[how to] Setting up cron and RSS to auto-generate newsletter

[kerberos]

Create administrator who is not "super" admin. Add name of user to commandline_users in config.php

Define tmpdir in config.php to /server/path/to/lists/tmp

Leave MANUALLY_PROCESS_RSS,1 and MANUALLY_PROCESS-QUEUE,1 in config.php

Also in config.php, set MAILQUEUE_BATCH_SIZE,250 and MAILQUEUE_BATCH_PERIOD,3600 and MAILQUEUE_THROTTLE,5 to send 250 messages per hour with 5 seconds between each one.

cronjobs:

[get rss]
php /server/path/to/lists/admin/index.php page=getrss config=/server/path/to/lists/config/config.php login=adminuser password=password
[
if getrss is run frequently (more often than every 3 hours, to help order items in the message, where they are ordered by when they were added by getrss, not by their original publication date), then reduce the cache time:
In admin/onyxrss/onyx-rss.php, change
$this->conf['cache_time'] = 180;
to
$this->conf['cache_time'] = 1;
]

[process queue]
php /server/path/to/lists/admin/index.php page=processqueue config=/server/path/to/lists/config/config.php login=adminuser password=password

RSS:

Define ENABLE_RSS,1 in config.php

Add RSS feed(s) to list.

Create RSS message template: include "[RSS]" in body.

In Format tab, select HTML.

In Scheduling tab, select frequency (which will be matched to user RSS frequency; this must be done separately for each frequency). Set the embargo time before cron jobs to get rss and process queue.

In Lists tab, select list(s) for receipt and send.

It should all be automatic now, with cronjobs getting the RSS and processing the queue. To ensure that a message goes out regularly (if there is new material in the RSS feed), set the minimum items to 0 on the administrative configuration page.

SMTP (better for HTML):

Define PHPMAILER,1 and PHPMAILERHOST,'localhost'

Leave $phpmailer_smtpuser and $phpmailer_smtppassword commented (and set to '' for security)

Miscellaneous fixes/adjustments:

In database:
• Allow NULL in to, replyto, from, and subject columns.
• Set default htmlemail to 1 and rssfrequency to daily.
• SQL query to set rssfrequency for all existing users:
UPDATE 'phplist_user_user' SET 'rssfrequency' = 'daily'
[This is simply to allow a single message (i.e., daily) and to ensure every subscriber is set to receive it.]

In admin/getrss.php:
Limit SQLqueries to 255 characters (0-254): Change
Sql_Query(sprintf('select * from %s where title = "%s" and link = "%s"', $tables["rssitem"], addslashes($item["title"]), addslashes($item["link"])));
to
Sql_Query(sprintf('select * from %s where title = "%s" and link = "%s"', $tables["rssitem"], substr(addslashes($item["title"]),0,254), substr(addslashes($item["link"]),0,254)));

In rssitem table:
Set link and title fields to 255 chars (corresponding to above change).

[Guest]

Hey kerberos (and everyone):

This is exactly what I'm trying to do with my list. I'm con-fussed by your instructions though.

Help?

cronjobs:

[get rss]
php /server/path/to/lists/admin/index.php page=getrss config=/server/path/to/lists/config/config.php login=adminuser password=password

[process queue]
php /server/path/to/lists/admin/index.php page=processqueue config=/server/path/to/lists/config/config.php login=adminuser password=password

I've been playing with cron and was having trouble with your coding above. With help from another thread, I got this to work:

Code: Select all

/usr/local/bin/php /home/username/public_html/newsletter/admin/index.php -pgetrss -c/home/username/public_html/newsletter/config/config.php

/usr/local/bin/php /home/username/public_html/newsletter/admin/index.php -pprocessqueue -c/home/username/public_html/newsletter/config/config.php



Hope that helps someone.


I haven't attempted the sql queries yet, and I'm frankly nervous about them.

The main problem I'm encountering is that the messages say they're being sent, but I'm not receiving anything. The RSS is being cron'd okay, and the system is getting them; the process queue is croning okay as well. But I'm just not getting anything in email.

Is this a part of what your sql hacks are for?

MTIA. I'm stoked to get this working.

[kerberos]

One of the mysql hacks is simply to make sure that every subscriber is set to daily, so I don't have to worry about creating any messages other than daily.

The other hack, including the phplist mysql query, lengthens the fields for comparing newly fetched rss items with earlier items already in the database (to make sure they're new).

[yeahdisk]

hi

My shared host doesn't allow for Cronjobs to be run on their server.

I wanted to test th cron using webcron.org, but this asks for cronjobs to start with http:// ???

How would I translate the Cronjobs in this thread to that specific format?

Please help I've never worked with cron before!

[yeahdisk]

Also, would it be possible to combine these two actions (Get_RSS and Process_Queue) into a single .php file so I can just call that file using a single cron job?