You are here:  » Cron Job always active


Cron Job always active

Submitted by Tobix on Sun, 2020-12-27 06:58 in

Hi, I noticed that my server's CPU was always going to 100%. I deactivated the WP plugins and had my hosting run an analysis and they noticed that I had a problem with the Cron Job. How can I solve?

Submitted by support on Sun, 2020-12-27 08:39

Hello Tobix,

I think this may be related to your other question regarding import so let's cover them both here - firstly I wanted to check, when you installed the plugin did you change to the Imported option (where actual WordPress posts are created for each product), or did you leave it as "Virtual"?

If you did change to "Imported", did you also enable CRON on the PriceTapestry.org settings page and in which case what settings did you choose; could you perhaps temporarily change back to "Virtual" just as a test to see whether it is the Imported mode that has caused the problems?

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Sun, 2020-12-27 14:41

I am in virtual mode I can do the reverse see what happens by setting the Imported mode!

Submitted by Tobix on Sun, 2020-12-27 15:27

My settings are still "Virtual" and the Cron command is:
{link saved}

Submitted by support on Tue, 2020-12-29 07:59

Hi Tobix,

I would stay in Virtual mode whilst investigating server load issues. Can you tell from the time that you have your Price Tapestry cronjob scheduled to run vs the Modified/Imported times as shown on the /pt/admin/ home page how long cron.php is taking to run?

How many feeds / total products are in the installation?

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Tue, 2020-12-29 21:17

cron can take 20 minutes.
I have around 20 feeds and 315,000 products.
Now the cron is disabled and I am manually updating the feeds :(

Submitted by support on Wed, 2020-12-30 07:57

Hi Tobix,

I would suggest applying the 1 second sleep after every 100 products during import as described in this thread which significantly reduces processor load during cron - in this case it will increase the time by almost 1 hour but should reduce load by 80%...

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Wed, 2020-12-30 16:34

problem solved! great!
How can I activate the "slow import" now? Does not work :(

Submitted by support on Thu, 2020-12-31 10:06

Hi Tobix,

What happens when you run the Slow Import Tool for a feed? e.g. does it appear to attempt the first block (500 products) and then finish without completing? This is actually a complex issue to do with the way certain XML parser libraries used by PHP (it's not necessarily the same depending on operating system) so in some cases the tool is really only suitable for CSV feeds.

Is the problem trying to use normal Import function from /pt/admin/ just that it times out in the web browser window (as sometimes modifying the script to generate output helps so let me know if that's the case...)

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Thu, 2020-12-31 11:13

Yes, that's it. The scan of the first 500 starts and then everything stops.

Submitted by support on Thu, 2020-12-31 13:39

Hi Tobix,

If it's an XML feed that is problem the library incompatibility issue. What happens when you try a normal (not slow) Import for an individual feed - do they complete successfully or do they timeout which is why you are trying to use Slow Import?

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Thu, 2020-12-31 13:44

It depends. Very often the process is completed but sometimes not and it goes to the admin login screen

Submitted by support on Fri, 2021-01-01 09:17

Hi Tobix,

It might be worth trying the mod to make manual import generate output; to do this, edit admin/feeds_import.php and look for the following code at line 25:

    admin_import($filename,$limit);

...and REPLACE with:

    admin_import($filename,$limit,"admin_importCallback");

And then the following code at line 37:

          admin_import($feed["filename"],0);

...and REPLACE with:

          admin_import($feed["filename"],0,"admin_importCallback");

And finally the following code at line 43:

  admin_importReviews();

...and REPLACE with:

  admin_importReviews();
  function admin_importCallback($count)
  {
    print $count."<br />";
  }

Cheers,
David.
--
PriceTapestry.com

Submitted by Tobix on Fri, 2021-01-01 16:23

Error:
{code saved}

Submitted by Tobix on Sat, 2021-01-02 07:40

unfortunately the CPU started 100% again

Submitted by support on Sat, 2021-01-02 09:23

Hi,

Ah sorry - admin/import.php would need to use meta refresh rather than header() to redirect back to the Admin home page. To do this, look for the following code just before the exit() statement at the end of the script;

  header("Location: ".$config_baseHREF."admin/");

...and REPLACE with:

  print "<meta http-equiv='refresh' content='0;url=".$config_baseHREF."admin/' />";

So regarding CPU 100%, I wonder if it's not actually cron related at all; maybe aggressive robots crawling product pages too quickly perhaps. You could try adding to the end of your top level robots.txt

User-agent: *
Crawl-delay: 2

Cheers,
David.
--
PriceTapestry.com