Throttled Sending Instead of Batches

Solutions for other advanced phplisters

Solution to cron problem!

Postby sharper » 4:29pm, Sat 22 Jan, 2005

I have set up the throttle crontrol as above and it seems to work well, although as my site is hosted by Ipowerweb and access to cron setup is limited (and support worse), I was having difficulty getting it to run as a cron job.

I have however found the following site http://www.webbasedcron.com who will actually process the cron jobs remotely. It's only $8.95 per annum.

I've been using it on a very small test list and it seems to work, so others may like to give it a try if you're not having any success with your hosting company.
sharper
phpLister
 
Posts: 5
Joined: 7:11am, Tue 11 Jan, 2005

Postby Tae » 5:59pm, Mon 24 Jan, 2005

i'm new to phplist and crons (tho i understand what it does). i have a list of 3000 members so i'd imagine that a cron makes most sense. but i'm having a difficult time with where to start. i believe that my host is very flexible with crons (i have shell access but have never used it) so should i look into cron myself or is there a benefit to a service such as webbasedcron.com? $8.95 is very reasonable if it makes things easier even if i have full cron capability.

o, and is there a simple, abc's of how to setup a cron for phplist for ppl that have full cron capability? thanks.
Tae
phpList newbie
 
Posts: 1
Joined: 5:49pm, Mon 24 Jan, 2005

Webbasedcron.com

Postby sharper » 6:23pm, Mon 24 Jan, 2005

If your host can accomodate you - then there should be no problem.

Webbasedcron.com is only likely to be an advantage to those (like myself) who can't get diddly from their host :(

I have used the throttling method and it works well - ipowerweb have a riddiculously low limit on outgoing emails, so I set the throttle to 3 secs and they didn't notice a thing. Slow for a large list though, which mine isn't!

S
sharper
phpLister
 
Posts: 5
Joined: 7:11am, Tue 11 Jan, 2005

Postby photofx » 7:02am, Wed 26 Jan, 2005

I have found a work around for PHP in CGI mode, as quoted below by ryanc (thanks boss). It seems to work fine though I have only tested with a small list of 42 (time to send 8min 44sec). The real test will come when it goes out to 3000 (or more for some of you).

ryanc wrote:FWIW, I have this line setup in my cron to run every hour.. Seems to work fine, and I didn't modify any php files. Maybe it will help someone.

Code: Select all
wget -q  -O - 'http://mywebsite.com/phpList/admin/index.php?page=processqueue&login=myusername&password=mypassword' > /dev/null


Ryan


I have also implimented drubarth's Throttle (top of this thread) as I have a server limit of 300 mails per hour. I have set the throttle to pause 12 sec between each mail which by my calculation sends 300 mails in an hour. My smallish list of 3000 therefore would take some 9-10 hours to send.

My question is: What frequency should I set my cron job? I.E Does it reset the queue each time the cron comes into effect? How does this effect the sending of the mail?

If I don't use the cron job and use manual send by clicking "process queue", will the browser timeout? Can I just close my browser after clicking process queue? To my mind this wont work, therfore I would prefer to use the cron.

Anyone have any ideas on this?

Kevin
photofx
phpLister
 
Posts: 6
Joined: 7:16am, Wed 17 Nov, 2004

Throtle v Batch Processing

Postby sharper » 1:11pm, Wed 26 Jan, 2005

First off - I'm no expert!

However, I think you will find that you can sneak a whole lot more emails through your host than your are officially allowed (300/hour) in your case, using the throttle method - I was able to do more that ipowerweb allow - a lot less than 300/hour. Try reducing the gap to 2 or 3 seconds and see if it works.

Second, when using throttle you don't nee to increase the frequency of the cron. Once the job starts, that's it - but the issue may be if there is a timeout set on the cron job - it needs to be long enough to allow your job to finish. On that basis you can set the cron to run just once or twice daily (or whatever suits you) unlike batch where I think you need to schedule it to coincide with your batch limit timings.

Hope this helps.
sharper
phpLister
 
Posts: 5
Joined: 7:11am, Tue 11 Jan, 2005

Postby photofx » 1:29pm, Wed 26 Jan, 2005

Thanks for that Stephen, I am sure you are right.

I have just sent off the list to 3000 odd members with the cron set to 1 hourly checks, and still the 12 sec delay.

On the next Weekly mail I will try your suggestions and post the results. Client was a bit upighty today about members not receiving their mail so I thought I'd start with the widest parameters and work from there.

regards
Kevin
photofx
phpLister
 
Posts: 6
Joined: 7:16am, Wed 17 Nov, 2004

Postby larson » 8:47pm, Fri 28 Jan, 2005

I'm not trying to be a smartass but I was a bit confused about how to convert seconds into milliseond and I looked up the meaning and it said that a millisecond is "one thousandth of a second". I think a millionth of a second is called a microsecond. I guess my question would be how do I write half a second for this script? Is it 500 or 500000? Thanks!
-Larson
larson
PL Nut
 
Posts: 31
Joined: 10:22pm, Mon 26 Apr, 2004

Postby rkdesantos » 10:54pm, Sat 29 Jan, 2005

larson wrote:I'm not trying to be a smartass but I was a bit confused about how to convert seconds into milliseond and I looked up the meaning and it said that a millisecond is "one thousandth of a second". I think a millionth of a second is called a microsecond. I guess my question would be how do I write half a second for this script? Is it 500 or 500000?


Milli = from Latin "mille" or "thousand" Therefore millisecond = thousandths of a second. "million" is a thousand thousand or 1,000,000.

So 0.5 seconds = 500 milliseconds. Clear? :-)
rkdesantos
PL Nut
 
Posts: 21
Joined: 5:58am, Sun 22 Feb, 2004

Postby larson » 5:38pm, Mon 31 Jan, 2005

perfect! thank you!!
-larson
larson
PL Nut
 
Posts: 31
Joined: 10:22pm, Mon 26 Apr, 2004

Postby SubhenduSen » 7:45am, Wed 09 Mar, 2005

Hi...

I've made the following changes according to suggestions of drubarth :

In config/config.php :

Code: Select all
# define the amount of emails you want to send per period. If 0, batch processing
# is disabled
define("MAILQUEUE_BATCH_SIZE",0);

# Throttle sending messages; indicates the number of milliseconds to pause
# between each message (millisecond is one-millionth of a second)
# As suggested at : http://www.phplist.com/forums/viewtopic.php?t=2142&postdays=0&postorder=asc&start=0
define("MAILQUEUE_PROCESSING_THROTTLE",2000);

# define the length of one batch processing period, in seconds (3600 is an hour)
define("MAILQUEUE_BATCH_PERIOD",3600);


In admin/processqueue.php :

Code: Select all
            if (VERBOSE)
              output("Sending $messageid to $useremail");
            sendEmail($messageid,$useremail,$userhash,$htmlpref,$rssitems);
         # refer: http://www.phplist.com/forums/viewtopic.php?t=2142&postdays=0&postorder=asc&start=0
          usleep(MAILQUEUE_PROCESSING_THROTTLE);
          } else {
            sendEmailTest($messageid,$useremail);
          }


I have the following line in cronjob :

Code: Select all
*/15 * * * *   lynx -dump 'http://MYDOMAIN.COM/FOLDER_TO_PHPLIST/admin/index.php?page=processqueue&login=USERNAME&password=PASSWORD' > /dev/null


I use CPanel and have this cronjob set in cpanel.

After I reque a message last sent and processed, I get following message :

Code: Select all
Started
Processing has started, 1 message(s) to process.
It is safe to click your stop button now, report will be sent by email to YOU@YOURDOMAIN.COM
Processing message 23
Looking for users
Found them: 499 to process
It took very little time to send this message
Script stage: 6
Finished, Nothing to do
0 messages sent and 497 messages skipped
2 emails unconfirmed (not sent)
Finished this run
Finished, All done


I haven't received any mail from the list.

What's the error? Can someone help please?

Thanks in advance

Subhendu
SubhenduSen
phpLister
 
Posts: 8
Joined: 7:32am, Wed 09 Mar, 2005

can you use batch processing and throttling together?

Postby artzpimp » 2:03am, Thu 10 Mar, 2005

I'm using ipowerweb and am limited to 5 emails per 10 seconds and a max of 100 per hour.

My problem is that it doesn't seem to send multiple emails. When I create the test list with 2 or 4 emails it will send properly. when I go to a larger list of 37 is skips them.

I tried setting the throttling to 45000 or 45 seconds between emails - still no luck. I've set the batches to 95 emails per hour.

I inserted the throttling script after the batch script

Is my problem that i am combining batches and throttling?

BTW I'm very new to php

Thanks in advance
artzpimp
phpList newbie
 
Posts: 3
Joined: 1:52am, Thu 10 Mar, 2005

Postby SubhenduSen » 4:29am, Fri 11 Mar, 2005

I have skipped throttling and went for batch processing with multiple cronjobs and thank god it's working now.

Please read this post of Karma : http://www.phplist.com/forums/viewtopic.php?t=2146 , follow the steps, replace the values which suits you...and you are done.

Hope this helps...

Cheers

Subhendu
SubhenduSen
phpLister
 
Posts: 8
Joined: 7:32am, Wed 09 Mar, 2005

Postby silverbytes » 4:39am, Mon 18 Jul, 2005

Hi drubaru

I changed both files you mention adding the code and have cli. But I don't know what to do next. Can I enter cpanel and set a cron job there? What should I enter to execute sending a message to a determinated list?
silverbytes
PL Nut
 
Posts: 45
Joined: 4:13pm, Fri 18 Mar, 2005

Postby drubarth » 6:05pm, Mon 18 Jul, 2005

You can test it by Telnet/SSHing into the server and entering the commands. Once you have the commands working, then setup your cron job.
Darryl Rubarth

Web hosting and support for open-source solutions
www.exyst.com
drubarth
PL Master
 
Posts: 188
Joined: 10:14pm, Fri 17 Dec, 2004

Cron and batch

Postby silverbytes » 5:29am, Wed 20 Jul, 2005

Tried all mentioned but something is not ok.
I have a test list active with 3 emails.
config.php is like this (1 email per mintue since I only have a list of 3 emails to test, I need to test the cron job to ensure it's working, I'll tell to send 1 email per mintue max, and 1 cron every 5 minutes, so in 15 mins all 3 messages should be sent.

Code: Select all
# define the amount of emails you want to send per period. If 0, batch processing
# is disabled
define("MAILQUEUE_BATCH_SIZE",1);

# define the length of one batch processing period, in seconds (3600 is an hour)
define("MAILQUEUE_BATCH_PERIOD",60);


My cron job from cpanel is set:

php /home/mysite/public_html/phplist/admin/index.php page=processqueue login=mylogin password=mypass>/dev/null

Went to messages, clicked send message with test list checked:

But I see this in messages Sent:
1 messages

Message info Status Action
From: me@mysite.com
Subject: Envío de prueba3
Entered: 2005-07-20 00:56:37
Embargo: 2005-07-20 02:00:00
inprocess

view
2 done
edit
delete



The eventlog shows:

Code: Select all
210 del processqueue 2005-07-20 02:20:00 Finished this run
 
199 del processqueue 2005-07-20 02:15:00 Processing has started, 1 message(s) to process.
 
200 del processqueue 2005-07-20 02:15:00 It is safe to click your stop button now, report will be sent by email to bla@bla
 
201 del processqueue 2005-07-20 02:15:00 Processing message 4
 
202 del processqueue 2005-07-20 02:15:00 Looking for users
 
203 del processqueue 2005-07-20 02:15:00 Found them: 3 to process
 
204 del processqueue 2005-07-20 02:15:00 Finished this run
 
193 del processqueue 2005-07-20 02:10:00 Processing has started, 1 message(s) to process.
 
194 del processqueue 2005-07-20 02:10:00 It is safe to click your stop button now, report will be sent by email to bla@bla
 
195 del processqueue 2005-07-20 02:10:00 Processing message 4
 
196 del processqueue 2005-07-20 02:10:00 Looking for users
 
197 del processqueue 2005-07-20 02:10:00 Found them: 3 to process
 
198 del processqueue 2005-07-20 02:10:00 Finished this run
 
186 del processqueue 2005-07-20 02:05:01 Processing has started, 1 message(s) to process.
 
187 del processqueue 2005-07-20 02:05:01 It is safe to click your stop button now, report will be sent by email to bla@bla
 
188 del processqueue 2005-07-20 02:05:01 Processing message 4
 
189 del processqueue 2005-07-20 02:05:01 Looking for users
 
190 del processqueue 2005-07-20 02:05:01 Found them: 3 to process
 
191 del processqueue 2005-07-20 02:05:01 No users to process for this batch
 
192 del processqueue 2005-07-20 02:05:01 Finished this run
 
185 del processqueue 2005-07-20 02:00:00 Finished this run



What is happening really???
silverbytes
PL Nut
 
Posts: 45
Joined: 4:13pm, Fri 18 Mar, 2005

PreviousNext

Return to Advanced Answers, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 1 guest

cron