Resorting mailing list

Questions about phpList features (the answer is probably 'Yes!')
Forum rules
Please help the volunteers to help you by supplying the version of phpList you are using, browser & version and if possible, a link to your phpList installation. This is for Version 2 of phpList (the orange one).

Resorting mailing list

Postby Helveticus » 9:31am, Thu 15 Aug, 2013

Hi

Is there a possibility that phpList does resort my mailing list somehow before sending? So for example if I have a lot of mail addresses with @yahoo.com then they should not be sent all in sequence.
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 5:32pm, Thu 15 Aug, 2013

Yes it does! ( at least in v2.11.10 & v3.0.0 ) :wink:

If you open your 'config/config.php' file, you will find starting on line 358, DOMAIN THROTTLING.
Code: Select all
# Domain Throttling
# You can activate domain throttling, by setting USE_DOMAIN_THROTTLE to 1
# define the maximum amount of emails you want to allow sending to any domain and the number
# of seconds for that amount. This will make sure you don't send too many emails to one domain
# which may cause blacklisting. Particularly the big ones are tricky about this.
# it may cause a dramatic increase in the amount of time to send a message, depending on how
# many users you have that have the same domain (eg hotmail.com)
# if too many failures for throttling occur, the send process will automatically add an extra
# delay to try to improve that. The example sends 1 message every 2 minutes.

define('USE_DOMAIN_THROTTLE',0);
define('DOMAIN_BATCH_SIZE',1);
define('DOMAIN_BATCH_PERIOD',120);

Alternatively, you could use the AUTO DOMAIN THROTTLING starting on line 372:
Code: Select all
# if you have very large numbers of users on the same domains, this may result in the need
# to run processqueue many times, when you use domain throttling. You can also tell phplist
# to simply delay a bit between messages to increase the number of messages sent per queue run
# if you want to use that set this to 1, otherwise simply run the queue many times. A cron
# process every 10 or 15 minutes is recommended.
define('DOMAIN_AUTO_THROTTLE',0);

Hope that helps! :wink:

Sven2157
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 9:52pm, Thu 15 Aug, 2013

Thank you very much.

I don't understand the description of define('DOMAIN_AUTO_THROTTLE',0);.

What does it do exactly? Can I use it together with USE_DOMAIN_THROTTLE ?

Which one would you reccommend?


By the way. I have set in the config.php that only 10'000 mails should be sent each 90'000 seconds. I'm using a cronjob which calls the processque every 20min. Now, When the 10'000 mails are sent but the 90'000 seconds are not over and the cronjob calls the processque I get everytime a mail saying that no more sending is possible in the current period.

How can I stop that?
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 11:34pm, Thu 15 Aug, 2013

Helveticus wrote:I don't understand the description of define('DOMAIN_AUTO_THROTTLE',0);.
What does it do exactly? Can I use it together with USE_DOMAIN_THROTTLE ?
Which one would you reccommend?

You are correct that this a bit under-written; or not well explained. However, I believe that the DOMAIN_AUTO_THROTTLE is a function that will try to automatically determine settings for the DOMAIN_BATCH_SIZE and DOMAIN_BATCH_PERIOD.

Though reading the description for the DOMAIN_BATCH, it would seem that AUTO_THROTTLING will turn itself on, automatically, if there are too many errors with sending out your emails. I would try the DOMAIN_BATCH alone first. If that doesn't work well, or you can't figure out a good setting to use, you can turn the AUTO_THROTTLE on, and see how that goes.
Helveticus wrote:By the way. I have set in the config.php that only 10'000 mails should be sent each 90'000 seconds. I'm using a cronjob which calls the processque every 20min. Now, When the 10'000 mails are sent but the 90'000 seconds are not over and the cronjob calls the processque I get everytime a mail saying that no more sending is possible in the current period.

How can I stop that?

10,000( ten thousand ) emails? :shock: 90,000( ninety thousand ) seconds? :shock: That's like 62.5 days ... :| That's only 160 emails a day. Your Cron Job is waiting for that time period to end. So every time your Cron runs, it sees that it is still sending, and that's why you get that email about not being able to send.

You should set the amount to go out, to something safer, like 500 every hour( 3600 seconds ). It is important to know and understand what your hosting will allow you to send an hour, or they may shut you down.

Having said that, it really doesn't matter what is set, as when the cron job tries to run again, and the process queue is already processing, you will receive that email. You can turn off the email notification from the Cron Job by adding '> /dev/null 2>&1' to the end of the cron task script. You can then turn on notification in phplist for when the Queue Processing starts, and another for when it finishes.

Hope that helps! :wink:

Sven2157
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 12:30pm, Fri 16 Aug, 2013

90'000 seconds are 25 hours.

90'000 / 60 / 60 = 25.

Or I am wrong?
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 10:30pm, Fri 16 Aug, 2013

Helveticus wrote:90'000 seconds are 25 hours.

90'000 / 60 / 60 = 25.

Or I am wrong?

Ha! :lol: No you are correct; not sure how I got those hours :oops: My math skills are what kept me out of Clown College!

...

However, that is still only 400 emails in a 25 hour period.
  • 10,000 emails/25 hours = 400
I personally, would break that down to an total/hour:

Code: Select all
# batch processing disabled:
define("MAILQUEUE_BATCH_SIZE",400);

# Batch_period is not effective when batch processing is disabled:
define("MAILQUEUE_BATCH_PERIOD",3590);

# Pause between messages (in seconds) to send no more than 360 messages per hour:
define('MAILQUEUE_THROTTLE',10);


If anything, you will give your server processor/RAM a bit less to chew on ...

Sven2157
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 11:02am, Sat 17 Aug, 2013

But Amazon SES allows only 10'000 mails per 24h...
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 2:48pm, Sat 17 Aug, 2013

Helveticus wrote:But Amazon SES allows only 10'000 mails per 24h...

Ok? :| So
  • 400( emails ) x 24( hours ) = 9600 emails sent an hour, in one day.
You could actually set the emails to 416:
  • 416( emails ) x 24( hours ) = 9984 emails sent an hour, in one day.

When you use smaller numbers, it is easier to read by humans, there is less of a chance to make fundamental mistakes( as I have already proved with the 62.5 days, thing), and it uses less memory( RAM ) and processing( CPU ), on the server.

Me personally? ... With a 100k list like yours, I would NOT want to do anything "extra", as a list that size, could take weeks to hunt down issues ...

:wink:

Sven2157
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 6:49pm, Sat 17 Aug, 2013

Thank you again.

What throtteling value would you use?

I'm currently using

Code: Select all
define('USE_DOMAIN_THROTTLE',1);
define('DOMAIN_BATCH_SIZE',2);
define('DOMAIN_BATCH_PERIOD',60);


So to a domain only 2 messages per 1 hour is sent.
Last edited by Helveticus on 2:51pm, Tue 20 Aug, 2013, edited 2 times in total.
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Helveticus » 10:19am, Tue 20 Aug, 2013

Can somebody help? Pleeeeease. :)
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 11:29pm, Tue 27 Aug, 2013

Sorry man, my computer died .. I have spent the last 5 days trying to desperately recover it.

I don't use domain throttling, but if I were sending to a list the size of yours, I would try this:
Code: Select all
define('USE_DOMAIN_THROTTLE',1);
define('DOMAIN_BATCH_SIZE',200);
define('DOMAIN_BATCH_PERIOD',3600);  <-- hour


I would see how that goes ...

Also on a side note, I saw your other posts about email validation. You haven't replied to a couple questions I had, yet. Just to let you know, I am working on an eMailValidator. This will check 2 things: if the email addresses' domain exists, and then whether the actual email address exists, within the domain.
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 12:08pm, Sun 01 Sep, 2013

Thank you.

To which questions I have not replied yet? I'm not aware of this.

Such an eMailValidator would be great. When do you expect to finish?
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby Sven2157 » 1:03pm, Sun 01 Sep, 2013

Helveticus wrote:Thank you.

To which questions I have not replied yet? I'm not aware of this.

Such an eMailValidator would be great. When do you expect to finish?

I was referring to this post: Importing list of blacklisted users.

I am currently working on another at the moment, and need to think this one through a bit more. I would say, maybe another couple days to a week... If you're interested in testing it out let me know.

How are those batch settings working out for you?

Sven2157
Sven2157
PL Master
 
Posts: 170
Joined: 12:18pm, Fri 07 Jun, 2013

Re: Resorting mailing list

Postby Helveticus » 9:30pm, Sun 01 Sep, 2013

I'm very interested in testing it out.

These batch settings seems to work for me, but I have to do a bit more testing and sending.
Helveticus
PL Geek
 
Posts: 64
Joined: 10:22pm, Sun 28 Jul, 2013

Re: Resorting mailing list

Postby zoeyku » 8:21am, Mon 23 Sep, 2013

I don't understand the description of define('DOMAIN_AUTO_THROTTLE',0);.
zoeyku
phpList newbie
 
Posts: 1
Joined: 8:16am, Mon 23 Sep, 2013

Next

Return to Does phpList do this?

Who is online

Users browsing this forum: No registered users and 2 guests