Making phplist able to use more than one smtp-Host

Technical discussion for developers and service providers

Making phplist able to use more than one smtp-Host

Postby lygie » 10:36am, Wed 08 Sep, 2010

Hallo,

some time ago I posted some code in the german forum
that makes it possible to use more than one relay-host to send e-Mails

viewforum.php?f=22

I am able to send several tenthousands emails within one our without problems.

Every update of phplist i have to rebuild my code on all my phplist installations.
I think it would be possible to modify the code and put this feature in phplist without problems.

I have tried to clearly arrange the code on my own website.
http://foobar.lamp-solutions.de/howtos/programmierung/einzelansicht-programierung/article/phplist-e-mails-ueber-mehrere-server-versenden.html

If there are any questions, or improvements, i will be glad to develop this feature.
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby H2B2 » 8:20pm, Thu 09 Sep, 2010

Thanks! Have opened the following feature request on this topic: http://mantis.phplist.com/view.php?id=15538
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: Making phplist able to use more than one smtp-Host

Postby TechLady » 7:57am, Fri 10 Sep, 2010

WOW... lygie this is great. :D ..trying it out right away. I am a website designer and am only a novice at php coding but you seem to have made it simple to follow.

I had a few questions as to where to place some of the codes. I think the numbers in the English language files may be different than your version. I am using Editpad lite to read and write the files and in my English verision but I also looked at them in alleycode with the same results.

The sendemaillib.php //line 23 is actually //line22, and the //line609 was //line606. Because you had the old and the new I was able to figure out what to do, thanks for that, but some of them do not have the old version so I am not sure exactly where to place them. You may want to add old {place between here and here}.

My first problem is with:
processqueue.php

Code: Select all
//line 352
      $hosts_count = sizeof($smtparray);
      $host_divide = 60/$hosts_count;

//line 641
      $date_s = date("s");
      $host_nr = floor ($date_s/$host_divide);
      $smtphost = $smtparray[$host_nr];

//line 647
      //old:
      $success = sendEmail($messageid,$useremail,$userhash,$htmlpref,$rssitems);
      //new:
      $success = sendEmail($smtphost,$messageid,$useremail,$userhash,$htmlpref,$rssitems);



Your //Line 647 is the easy one to track down because you have the old text, that text is at //Line633 in my version. That is a big seperation, 14 lines difference, so I am not sure about the other two segments. Where do they go?

This is how it looks in my 2.10.12 English version (these are the numbers shown in EditPad lite and Alleycode):
Code: Select all
352              while ($message = Sql_fetch_array($messages)) {
353               $failed_sent = 0;
354               $throttlecount = 0;



629            if (!$throttled) {
630              if (VERBOSE)
631                output($GLOBALS['I18N']->get('Sending').' '. $messageid.' '.$GLOBALS['I18N']->get('to').' '. $useremail);
632              $timer = new timer();
633              $success = sendEmail($messageid,$useremail,$userhash,$htmlpref,$rssitems);
634              if (VERBOSE) {
635                output($GLOBALS['I18N']->get('It took').' '.$timer->elapsed(1).' '.$GLOBALS['I18N']->get('seconds to send'));
636              }
637            } else {
638              $throttlecount++;
639            }
640          } else {
641            $success = sendEmailTest($messageid,$useremail);
642          }
643          if ($success) {
644            if (USE_DOMAIN_THROTTLE) {
645              list($mailbox,$domainname) = explode('@',$useremail);
646              if ($domainthrottle[$domainname]['interval'] != $interval) {
647                $domainthrottle[$domainname]['interval'] = $interval;
648                $domainthrottle[$domainname]['sent']=0;


Also, in the config.php I insterted the code after the ?> I hope that was correct.

Again, I am really looking forward to this working!!
TechLady
phpLister
 
Posts: 11
Joined: 5:36am, Tue 15 Sep, 2009

Re: Making phplist able to use more than one smtp-Host

Postby lygie » 7:39pm, Sun 12 Sep, 2010

Hallo TechLady,

>14 lines difference, so I am not sure about the other two segments. Where do they go?

The Line
Code: Select all
$success = sendEmail($messageid,$useremail,$userhash,$htmlpref,$rssitems);


calls the Funktion sendEmail, that actually performs the sending process.

The other two segments just have to be bevore this line. The must easiest thing would be to put it in front of the function call.

That should work any way.

If you want to do some performance tuning, you can put the segment.

Code: Select all
include_once ('smtphost.inc.php');
$hosts_count = sizeof($smtparray);
$host_divide = 60/$hosts_count;


Bevore the line where the script calls
Code: Select all
while ($message = Sql_fetch_array($messages)) {


So the two variables $host_count and $host_divide will only be set one time.

Good Luck,
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby TechLady » 12:19am, Tue 14 Sep, 2010

Thank you for the update. I have no idea how to read the code as to what it says and what it does. I am trying to learn but not there yet. And I would like to know. If you have the time I am sure there are others also who would like to know what each segment does. Maybe you could place it in German on your website and we could read it there with a translator. I know most of us are really busy so thanks again. Or rather "Dank sehr viel"
TechLady
phpLister
 
Posts: 11
Joined: 5:36am, Tue 15 Sep, 2009

Re: Making phplist able to use more than one smtp-Host

Postby TechLady » 1:04am, Tue 14 Sep, 2010

After adding all this code I am still confused as to where I tell it the server SMTP password and user, for each of the servers?
TechLady
phpLister
 
Posts: 11
Joined: 5:36am, Tue 15 Sep, 2009

Re: Making phplist able to use more than one smtp-Host

Postby lygie » 6:12am, Tue 14 Sep, 2010

Hallo TechLady,

> After adding all this code I am still confused as to where I tell it the server SMTP password and user, for each of the servers?

the most easiest thing would be to configure the SMTP-Host, that they allow to relay the E-Mails from phplist without a password.

That how i did it for my newsletter setups.

If this is not possible, you would have to modify

Code: Select all
$this->Username = $GLOBALS['phpmailer_smtpuser'];
$this->Password = $GLOBALS['phpmailer_smtppassword'];


in class.phplistmailer.php

Best would be, to put it in the configuration array and take it trough all funktion-calls.
I will try to get some ready-to-use code here. I guess it will take me about one day.

Yours sincerely,
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby TechLady » 7:29am, Sat 18 Sep, 2010

Thank you for the help I look forward to the ready to use code as I am really just a php beginner. :oops:
TechLady
phpLister
 
Posts: 11
Joined: 5:36am, Tue 15 Sep, 2009

Re: Making phplist able to use more than one smtp-Host

Postby jimbo » 3:06pm, Thu 23 Sep, 2010

lygie wrote:
> After adding all this code I am still confused as to where I tell it the server SMTP password and user, for each of the servers?

the most easiest thing would be to configure the SMTP-Host, that they allow to relay the E-Mails from phplist without a password.

That how i did it for my newsletter setups.

If this is not possible, you would have to modify

Code: Select all
$this->Username = $GLOBALS['phpmailer_smtpuser'];
$this->Password = $GLOBALS['phpmailer_smtppassword'];


in class.phplistmailer.php

Best would be, to put it in the configuration array and take it trough all funktion-calls.
I will try to get some ready-to-use code here. I guess it will take me about one day.




This is exactly what I need. I am on a shared server reseller account and I am limited to 350 emails per hour per cpanel account.
I cannot set the smtp to open relay.

How can i configure each accounts username and password, (maybe in array?) to send through multiple accounts like this?

Could an example be provided for maybe 3 domain names accounts?


Would something like this be a start? (at end of config.php)

Code: Select all
$smtparray = array("server1.tld", "server2.tld", "server3.tld");
$smtpuserarray = array("server1username","server2username","server3username");
$smtppassarray = array("server1password","server2password","server3password");
 if(!is_array($smtparray) && !is_array($smtpuserarray) && !is_array($smtppassarray)){
$smtparray[]=PHPMAILERHOST;
 }


Thank you so much or should I say Vielen Dank (I had to look that up). :wink:
jimbo
PL Nut
 
Posts: 15
Joined: 7:12pm, Wed 15 Sep, 2010

Re: Making phplist able to use more than one smtp-Host

Postby lygie » 8:49am, Sat 25 Sep, 2010

Sorry for the Delay in my answer. I was extremely busy the last week.

I have modified the code, so that you should be able to send passwords. The code is not tested yet, but is quite simple, so it should work.

At the end of the config file, the should be lines like

Code: Select all
#if you want do sent email over more than one smtp-Server, uncomment and fill following lines
$smtparray = array(
    "0" => array("hostname"=>"server1.tld","smtp_login"=>"username","smtp_passwd"=>"passwd"),
    "1" => array("hostname"=>"server2.tld","smtp_login"=>"username","smtp_passwd"=>"passwd"),
    "2" => array("hostname"=>"server3.tld","smtp_login"=>"username","smtp_passwd"=>"passwd"),
    );


The only thing that must be updated is class.phplistmailer.php

The part where hostname, passowrd, und username is defined should be:

Code: Select all
$this->Host = $mailhost["hostname"];
if($mailhost["smtp_login"]!=''){
  $this->Username = $mailhost["smtp_login"];
}
if($mailhost["smtp_passwd"]!=''){
  $this->Password = $mailhost["smtp_passwd"];
}


I hope, i will have time to test the code soon.
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby jimbo » 4:24pm, Sat 25 Sep, 2010

Excellent, Thank you lygie. :D
I will test.

The next item would be to set the from and reply in the header to match the settings.

That said, I had problems with many mails going into junk boxes and spam filters.
So, I have modified my php.ini for correct return path and envelope from which fixed a lot of that but, with your great mod, there would still be issues there.

Since the php.ini is specific to that domain, php.ini modded or un-modded, there will still be much junk fallout because the emails headers are not "correct" for the other smtp hosts. The settings in each phplist installation are static to the email they are set for in from & reply to. Maybe I am wrong but, there could only be one solution.

The only way to circumvent these issues, I think, would be to install a small mailer script on each domains account. Then, for load management, have the phplist send an email to that domains email which redirects to the script initiating the email from that domain with that domains credentials. Doing this, instead of having phplist send THROUGH different smtp servers would reduce junk mail fallout. This method would actually send the mail FROM that domain and smtp server using its' credentials, not the phplist domains credentials.

I am working on s script to do this. I believe, modding the class.smtp for installation on each domain would be a good way to do this. Even a perl formmail mod could be used.

Therefore, load management is far more effective. Your mod would just send an email to each domains email address which are each configured to redirect to a simple domain specific mailer.

So, when load management kicks in, it basically forwards the email to each other domains email, which then redirects to the script, which splits it up, puts the message subject and body into an new email and sends from that server with valid header credentials!

Just because it is simple does not mean it is easy. :roll:

I will try to do this based on your mod.

Thanks for all the help! (Vielen Dank)
jimbo
PL Nut
 
Posts: 15
Joined: 7:12pm, Wed 15 Sep, 2010

Re: Making phplist able to use more than one smtp-Host

Postby lygie » 2:06pm, Sun 26 Sep, 2010

Hallo jimbo,

The next item would be to set the from and reply in the header to match the settings.


is this really necessary?
The Mailheader can keep as it is set in phplist. I already use phplist and use more than one smtp relay for more than one year and sent some million mails without any problem. If you relay over a public IP, postfix puts the relay path in the header of the mail. But none of the important field like "from" or "reply to" or "envelope sender" it touched by relaying the email.

Yours, lygie
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby jimbo » 12:25pm, Mon 27 Sep, 2010

Hello thank you Lygie,

What is happening, with my list of only 210,000, is many mails are going into junk folders.

I opened a ticket with my hosting provider and they concur.

When the email from/reply is "a.org" (as set in the phplist configuration) and the email is sent through "b.org" the mismatch is what causes the issue.

The reply address and from address are static in phplists configuration. When the message is relayed through another domain, the headers reflect that domain.

So, clients like windows livemail put them right in the junk box.

Sorry I may have not been clear in my previous explanation.

I really need to increase my output. I need all of my domains, (some on dedicated, some on shared) to process our quickly growing member list without telling our subscribers to check in their junk mail boxes.

Thanks again for being patient with this phplist newbie.

Jimbo
jimbo
PL Nut
 
Posts: 15
Joined: 7:12pm, Wed 15 Sep, 2010

Re: Making phplist able to use more than one smtp-Host

Postby lygie » 1:14pm, Sat 09 Oct, 2010

Hallo Jimbo,

I agree, that the mailheader contains all hosts that relayed the massage.

Bur if all stations in the chain are fine, no junkfilter should complain.

Most important is that:
a) Neither the ip, nor the domain that is used are blacklistet
b) all ips have a valid reverse lookup
c) the emails have a valid return path and invalid mailadresses are deleted

No provider or program should mark mails from relay-hosts that have this qualities.

Sincerely
lygie
PL Nut
 
Posts: 19
Joined: 11:17am, Tue 24 Jul, 2007

Re: Making phplist able to use more than one smtp-Host

Postby sjanny01 » 7:34am, Wed 26 Jan, 2011

Every update of phplist i have to rebuild my code on all my phplist installations.
I think it would be possible to modify the code and put this feature in phplist without problems.
sjanny01
phpList newbie
 
Posts: 1
Joined: 7:20am, Wed 26 Jan, 2011

Next

Return to Developers Space

Who is online

Users browsing this forum: No registered users and 1 guest