Cron Job with PHP cgi - batch mode - using wget issues

Solutions for other phpList version 2 users
Forum rules
Please do not ask questions here, this is for Solutions you have discovered or come across.

Cron Job with PHP cgi - batch mode - using wget issues

Postby phplistar » 3:24pm, Sat 28 Apr, 2007

I have tried all the cron job options in "cron job examples" in the documentation, the only option that worked was:
Code: Select all
wget --post-data "page=processqueue&login=USERNAME&password=PASSWORD" http://your.domain.com/lists/admin/

Now I am not quiet sure how to use this when sending emails in batch mode(batch size 400). My list is about 50k, what is the best option here, any suggestions on how to go about? With 50k creating a cron entry for each batch is not an option it seems. If you can suggest complete settings I would really appreciate it. (I also plan to use the domain throttle option).
phplistar
phpLister
 
Posts: 9
Joined: 4:00am, Tue 03 Apr, 2007
Location: USA

Re: Cron Job with PHP cgi - batch mode - using wget issues

Postby H2B2 » 7:58pm, Sun 29 Apr, 2007

phplistar wrote:With 50k creating a cron entry for each batch is not an option it seems.
Instead of using batch processing, you could use the mailqueue throttle instead to space messages out in such a way that no more than 400 messages are sent per hour.
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Postby phplistar » 10:23pm, Sun 29 Apr, 2007

Thanks for your reply, Just to be sure
here is what I have:
Code: Select all
define("MAILQUEUE_BATCH_SIZE", 400);
define("MAILQUEUE_BATCH_PERIOD",3600);
define('MAILQUEUE_THROTTLE',0);
define('MAILQUEUE_AUTOTHROTTLE',0);

define('USE_DOMAIN_THROTTLE',1);
define('DOMAIN_BATCH_SIZE',2);
define('DOMAIN_BATCH_PERIOD',60);
define('DOMAIN_AUTO_THROTTLE',1);

I have to change these to:
Code: Select all
define("MAILQUEUE_BATCH_SIZE", 0);
define("MAILQUEUE_BATCH_PERIOD",0);
define('MAILQUEUE_THROTTLE',0.1)
define('MAILQUEUE_AUTOTHROTTLE',0);

define('USE_DOMAIN_THROTTLE',1);
define('DOMAIN_BATCH_SIZE',2);
define('DOMAIN_BATCH_PERIOD',60);
define('DOMAIN_AUTO_THROTTLE',1);


Please advise if this is correct. Are my domain throttle settings also correct?

Thanks again!
phplistar
phpLister
 
Posts: 9
Joined: 4:00am, Tue 03 Apr, 2007
Location: USA

Postby H2B2 » 10:48pm, Sun 29 Apr, 2007

With regard to your common message processing settings, I would suggest you try these:
Code: Select all
# batch processing disabled:
define("MAILQUEUE_BATCH_SIZE", 0);

# Batch_period not effective when batch processing disabled:
define("MAILQUEUE_BATCH_PERIOD",3600);

# Pause between messages (in seconds) to
# send no more than 400 messages an hour:
define('MAILQUEUE_THROTTLE',9);

# Autothrottle disabled:
define('MAILQUEUE_AUTOTHROTTLE',0);

I am a bit mystified by this setting: define('MAILQUEUE_THROTTLE',0.1). Are you using a mod, or have you perhaps misunderstood the syntax for that setting?

I have no experience with the experimental domain throttle feature, so I can't give you advice on those settings. You might search the forum for "define('USE_DOMAIN_THROTTLE',1);" and see if anything useful comes up.
Anyway, it seems to me you do not run a big risk of getting blacklisted with a send rate of 400 msgs/hr.
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Postby phplistar » 11:46pm, Sun 29 Apr, 2007

I am sorry, I miscalculated the period between each message , 3600/400 = 9 makes sense. I realize you can not have values less than a second for the time between each message, which I don't need anyhow. Thanks for your advice on domain throttling as well, I will turn this off.
phplistar
phpLister
 
Posts: 9
Joined: 4:00am, Tue 03 Apr, 2007
Location: USA

Postby H2B2 » 11:53am, Mon 30 Apr, 2007

phplistar wrote:define('MAILQUEUE_THROTTLE',0.1)

Well, there is a mod that allows fractional values for the mailqueue throttle setting: http://forums.phplist.com/viewtopic.php?p=11182#11182

That's why I wanted to know whether you did implement that mod. Being able to use fractional values in the throttle setting can be useful to maximize the send rate on a fast server with no message limits. This doesn't apply in your case of course.
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: Cron Job with PHP cgi - batch mode - using wget issues

Postby osarusan » 1:25pm, Wed 02 May, 2007

H2B2 wrote:
phplistar wrote:With 50k creating a cron entry for each batch is not an option it seems.
Instead of using batch processing, you could use the mailqueue throttle instead to space messages out in such a way that no more than 400 messages are sent per hour.


I'm also having trouble with getting the cron jobs to work, and I was wondering what to do until I saw this post. That's a good idea -- just to set a throttle rather than using batch processing. I just have a simple question before I try to implement this:

If "MAILQUEUE_BATCH_SIZE" is equal to 0 (i.e. disabled), then will the 'MAILQUEUE_THROTTLE' not work? Or, can I leave batch size disabled and just set a throttle, and that will take care of everything?

Thanks!
osarusan
phpLister
 
Posts: 11
Joined: 8:52pm, Thu 25 May, 2006

Re: Cron Job with PHP cgi - batch mode - using wget issues

Postby H2B2 » 3:04pm, Wed 02 May, 2007

osarusan wrote:Or, can I leave batch size disabled and just set a throttle, and that will take care of everything?
Exactly.
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: Cron Job with PHP cgi - batch mode - using wget issues

Postby kaon » 9:12am, Fri 23 Nov, 2007

H2B2 wrote:
osarusan wrote:Or, can I leave batch size disabled and just set a throttle, and that will take care of everything?
Exactly.

If I use mail throttle as above, say 7 seconds, and batch-size 0, I understand that the mailrun will be completed in one go, no repeated processqueue is needed.
Is it OK if I continue to have a half-hourly cron job that hits the processqueue using lynx/wget?
Will any bad things happen?
If msgs are separated by at least 7 seconds, then I would be within my webhost's send-rate limit.

The reason why I like to retain the half hourly cron job... might be an irrational fear... in case the php script on the web server gets interrupted some how.
kaon
PL Nut
 
Posts: 36
Joined: 7:18am, Fri 23 Dec, 2005

Postby H2B2 » 12:41am, Mon 26 Nov, 2007

Yes, if you space messages apart by 7 seconds, you will send no more than 514 messages per hour (3600/7=514). This is an effective alternative to batch processing and IMO the preferred method to use if your server is running PHP-cgi, which involves a higher risk of getting timeouts.
Ref: http://docs.phplist.com/PhpListConfigSendRate
See also: http://docs.phplist.com/PhpInterfacesInfo

Using a cron to run queue processing every 20 or 30 minutes should be alright. If phpList detects that the current process is still running, the new process should fail gracefully. If you hit a timeout, then the new process should take over where the old process left off.
See also these forum posts by jotazzu:
http://forums.phplist.com/viewtopic.php?p=11369#11369
http://forums.phplist.com/viewtopic.php?p=13145#13145
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: Cron Job with PHP cgi - batch mode - using wget issues

Postby indidginus » 12:07pm, Mon 19 Apr, 2010

Hi guys,

Apparently the server I am using is php-cgi, however I don't understand what that means for me. The instructions here (http://docs.phplist.com/PhpInterfacesInfo) go over my head.... Can I not login as normal to http://www.deviantdidg.com/lists/admin & send my mailing list messages from there? The info mentions config.php not being loaded into environment etc, which has confused me! Also, I'm not clear on how to set up a cron job from my mailing list, which I believe is necessary for php-cgi? If anyone can advise that would be greatly appreciated. Many thanks.

Michael.
indidginus
PL Nut
 
Posts: 44
Joined: 11:27am, Sun 18 Apr, 2010
Location: Cape Town, South Africa.


Return to Answers, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 1 guest

cron