How to debug a cron job that runs the script, but the script does not seem to finish?

I have a cron job scrape.sh that look like this:

#!/bin/bash
touch rage
cd /etc/myproject/scraper
scrapy crawl foosite --set FEED_URI=../feeds/foosite.xml --set FEED_FORMAT=xml
scrapy crawl barsite --set FEED_URI=../feeds/barsite.xml --set FEED_FORMAT=xml

When it executes the file rage does get created and judging from my syslog it does run as root, so permissions shouldn't be a problem.

May  6 17:35:01 server CRON[10233]: (root) CMD (/etc/myproject/scraper/scrape.sh)
May  6 17:40:01 server CRON[17804]: (root) CMD (/etc/myproject/scraper/scrape.sh)

When I run scrape.sh it executes as expected and puts the foosite.xml file in the ../feeds directory, the directory exist and is empty when the cron jobs starts. What can I do to solve this issue?

If I were going to guess the problem it was an environment issue (e.g. scrapy is not in the path).
To debug, make sure your cron job is sending the standard out and standard error to a log file/and or syslog

Help debug a cron job that has the correct script path and works when it's triggered manually

I'm struggling trying to debug a cron job which isn't working correctly. The cron job calls a shell script which should unrar a rar file - this works correctly when i run the script manually, but for some reason it's not working via cron. I am using

How to create a cron job to run a script from the Web?

I have created a cron job in a server www.domain.com with CENTOS: */10 * * * * http://www.anotherdomain.com/cron/task.php but its returning: /bin/sh: http://www.anotherdomain.com/cron/task.php: No such file or directory How to make it works? */10 * *

What is the best way to write a cron job that runs a PHP file with URL parameters?

I'm trying to setup some cron jobs that run a PHP script every 3 hours. I have three jobs all hitting the same script, but with different parameters. Option 1 I've learned that I can run the script right on the server like this: * */3 * * * /usr/bin/

Run a cron job that runs other cron programs in MYSQL

My system runs some intensive cron jobs twice daily. These are lengthy scripts. Occasionally, an action is performed in my system that requires the cron job to run on demand. My idea is to create a master cron job that checks my database for a table

Define 2 cron jobs that run every 10 minutes but do not work on top of each other

I can setup a cron every ten minutes like this: */10 * * * * But I want to setup 2 cron jobs that won't run at the same time. Is it possible to make one run every 10 minutes like this: 12.00, 12.10, 12.20, 12.30, 12.40... And the other run: 12.05, 12

how to create a cron job to run a ruby script?

I want to create a cron job to run a ruby script. this is what i have put in the crontab. 2 * * * * ruby /home/mark/project/script.rb >> /home/mark/cronOutput.txt But its not running. I think there's some problem with the environment getting loaded

How to write a Cron job to run a simple php script?

I have a simply php script on my server and want it to be run every 2 minutes using a cron job. */2 * * * * http://mydomain.com/_adder.php I suspect the command syntax is wrong. Do I need to add a command before the script url? Another way to run the

How to add a class via jQuery where the class does not have a name

In jQuery, how do I select and add a class to a class when the class does not have a value? Here is some example code: <div class=""> <div class=""> <span class=""><i class=""></i></

How to fix the error: the type does not seem to implement microsoft.practices.servicelocation.iservicelocator?

I am new to MVC, i am following "PRO ASP.NET MVC 4 by Adam Freeman". I am currently working on its 6th chapter. In which i am learning how to use Ninject in MVC 4 for Dependency Injection. I have created the application as described in the book.

How to create a scheduled job that runs every minute in the apex sales force

I am planning to create a scheduled job using scheduled apex in salesforce. I know that we can write and run the job for every one hour. I would like to know if there is a way we can run the code for every minute instead of every one hour. I know the

Write a simple cron job to run the java class

How can I write a cron job start from stratch to run a java class or write a cron job class with embedded java code to run? and how can I set the timer to run every one minute (for example) that cron job? Note : totally begineer with linuxHere is a s

How to debug a non-reactive server when the profiler can not collect samples?

I have been having occasional problems with a server I wrote. It's in Clojure, but I don't think that matters, and we can pretend it's in Java. Anyway, it works fine for hours at a time, but goes into fits where it behaves very badly: all activity st

How to reload getitem so that when the key does not exist, can create it

Is it possible to create a key value that does not yet exist, reload the method __ getitem__? I try: class Dict_(dict): def __getitem__(self, key): if key in self: self[key] = {} return self[key] else: return self[key] I understand why there is a sta

How to create a stored procedure that returns a random number if it does not exist in the table

I Want To Create A Stored procedure That return A Random Number Between (11111,99999) Provided that the Number Should Not Exist In The Table I use This complicated Function to Do that But I Need To Convert it to Stored Procedure Function GiveRandomSt

Related Articles