multiple phpLists reading same sql server

Questions about phpList features (the answer is probably 'Yes!')
Forum rules
Please help the volunteers to help you by supplying the version of phpList you are using, browser & version and if possible, a link to your phpList installation. This is for Version 2 of phpList (the orange one).

multiple phpLists reading same sql server

Postby sintsu » 2:10pm, Wed 02 Nov, 2011

Hi, there

I'm wondering how to use many phplists to reading same sql server to send the messages.

I've tried use two phplists reading the same sql and use cron to read their own index.php
Code: Select all
*/2 * * * * /usr/local/bin/php /usr/local/www/lists/admin/index.php -pprocessqueue  > /dev/null
*/3 * * * * /usr/local/bin/php /usr/local/www/data/lists/admin/index.php -pprocessqueue  > /dev/null

But the result turns out that the queue still waiting the previous message finish the sending.
Code: Select all
phpList version 2.10.17 (c) 2000-2011 phpList Ltd, http://www.phplist.com
A process for this page is already running and it was still alive 1 seconds ago
Running commandline, quitting. We'll find out what to do in the next run.

I don't know whether enable both multi-thread apache and mysql will solve it or not.

My next step is to create two master-client mysql table and let two phplist reading different tables and synchronizing in a certain time.

Is there anyone ever doing such testing or there's other way can send messages from the same sql DB by more than one process at the same time ?
sintsu
PL Nut
 
Posts: 21
Joined: 5:42am, Fri 07 Oct, 2011

Re: multiple phpLists reading same sql server

Postby sedkooo » 9:32pm, Wed 07 Mar, 2012

I'm also interested in this. Have you managed to solve this? Or anybody else has any solution to run multiple processes at the same time? What about enabling to run two instances by modifying the logic of selecting running process from DB, so that two can have tag alive? Thank you.
sedkooo
phpList newbie
 
Posts: 3
Joined: 9:27pm, Wed 07 Mar, 2012

Re: multiple phpLists reading same sql server

Postby CS2 » 2:49pm, Thu 08 Mar, 2012

What benefit are you trying to achieve? You're not going to achieve greater performance and there are very good reasons it won't let you process queue more than once at a time for the *same* tables.

If you're trying to run processqueue for *different* tables in the *same* database (i.e., multi-client install), that's a different thing entirely and there is existing documentation for this.
CS2
PL Master
 
Posts: 216
Joined: 2:20am, Wed 04 Feb, 2009

Re: multiple phpLists reading same sql server

Postby sedkooo » 8:39am, Fri 09 Mar, 2012

I was thinking about sending multiple messages simultaneously. Like one process sending one message and another process sending another message. It will be on the same tables but I think there will be no conflict, because I force the process to work only on one message. Am I wrong? And please can you give here a link where can I find the documentation to multi-client install? Thanks a lot.
sedkooo
phpList newbie
 
Posts: 3
Joined: 9:27pm, Wed 07 Mar, 2012

Re: multiple phpLists reading same sql server

Postby sedkooo » 12:58pm, Mon 23 Jul, 2012

Ok, I made it on my own:) Here is what I did:

table sendprocess: added column messageid and checked
lib.php: added function multiprocessCheck,
- this function selects all running processes from db and sort them by checked,
- if running processes < MAX_RUNNING_PROCESSES return 0, else return messageid from oldest process
- if running processes < MAX_RUNNING_PROCESSES check if oldest process is active (same as in getPageLock)

result from multiprocessCheck goes to getPageLock, where I added messageid to the first select where it checks for running
processes. This way getPageLock checks if there is no other process running for the same message.

So what I did was adding new layer to the processqueue (multiprocessCheck) at the beginning, that pass messageid to getPageLock.
This way phplist is able to send multiple messages at the same time.
With this solution we are able to send 50 000 messages per hour (based on usermessage table and also checked with first and last email received into my mailbox). We are running 5 parallel processes at server with Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz and 8GB ram. It is dedicated server where is also exim installed. I am also logging exim queue size and there are no problems. Only with one local mail provider here in Slovakia (4000 msgs/hour limit).

My next goal is to send messages through multiple SMTP servers.

If you are interested in my solution I can post modified files somewhere.

Please let my know what you think about this solution.

Thank you very much, and good luck in next development. PHPList is GREAT.
sedkooo
phpList newbie
 
Posts: 3
Joined: 9:27pm, Wed 07 Mar, 2012

Re: multiple phpLists reading same sql server

Postby seanfulton » 10:15pm, Sun 29 Jul, 2012

sedkooo

I am very interested. Can you send me the files (or a link to them).

I started doing some work using pcntl_fork() to split off a process for each send loop. It a work in progress, but this sounds like a really effective approach.

Please let me how I can get your changes. I can send you what I have on the fork method if you would like to take a look.

Thanks in advance!

sean
seanfulton
phpLister
 
Posts: 11
Joined: 11:43pm, Tue 06 Jul, 2004

Re: multiple phpLists reading same sql server

Postby spanish69 » 3:12pm, Thu 18 Oct, 2012

sedkooo wrote:This way phplist is able to send multiple messages at the same time.
(...)
If you are interested in my solution I can post modified files somewhere.


Another interested here. Thanks, Manuel
spanish69
phpList newbie
 
Posts: 2
Joined: 4:31pm, Thu 10 May, 2012

Re: multiple phpLists reading same sql server

Postby cybexin » 1:35pm, Thu 09 May, 2013

Please share the updated code.

Thanks in advance.
cybexin
phpLister
 
Posts: 5
Joined: 1:32pm, Thu 09 May, 2013

Re: multiple phpLists reading same sql server

Postby mercivoilaplease » 4:37pm, Tue 21 May, 2013

sedkooo wrote:Ok, I made it on my own:) Here is what I did:

table sendprocess: added column messageid and checked
lib.php: added function multiprocessCheck,
- this function selects all running processes from db and sort them by checked,
- if running processes < MAX_RUNNING_PROCESSES return 0, else return messageid from oldest process
- if running processes < MAX_RUNNING_PROCESSES check if oldest process is active (same as in getPageLock)

result from multiprocessCheck goes to getPageLock, where I added messageid to the first select where it checks for running
processes. This way getPageLock checks if there is no other process running for the same message.

So what I did was adding new layer to the processqueue (multiprocessCheck) at the beginning, that pass messageid to getPageLock.
This way phplist is able to send multiple messages at the same time.
With this solution we are able to send 50 000 messages per hour (based on usermessage table and also checked with first and last email received into my mailbox). We are running 5 parallel processes at server with Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz and 8GB ram. It is dedicated server where is also exim installed. I am also logging exim queue size and there are no problems. Only with one local mail provider here in Slovakia (4000 msgs/hour limit).

My next goal is to send messages through multiple SMTP servers.

If you are interested in my solution I can post modified files somewhere.

Please let my know what you think about this solution.

Thank you very much, and good luck in next development. PHPList is GREAT.


I join the others asking if you can please share your files.

Unfortunately we are not all PHP programmers here ... it would help many.

___________________________________________________________________
Linux is only free if your time has no value. -Linus Torvalds Free hosting
mercivoilaplease
phpList newbie
 
Posts: 1
Joined: 4:31pm, Tue 21 May, 2013


Return to Does phpList do this?

Who is online

Users browsing this forum: No registered users and 2 guests

cron