v2.10.10 New daily RSS users get the whole of the database

Discuss, research, share, find, and solve bugs

v2.10.10 New daily RSS users get the whole of the database

Postby alanf » 5:24pm, Sun 19 Jul, 2009

I'm not sure technically if this is a bug, or a design feature that I don't think produces the right results. I'll err on the side of 'bug' and take any feedback that comes my way.

When running an RSS e-mail with daily frequency, and another with monthly, I need to keep at least 1 months worth of RSS data on the database.

If a NEW user subscribes to the list as 'daily', on the first day the new user will get everything held on the RSS tables on the first e-mail, as nothing has been flagged as being sent to that user.

In an ideal world, when a new user subscribes to a daily list, the RSS table would be flagged to excluded everything over a day old for that user, son on the first e-mail they get just the recent RSS data (and the same logic for weeky / monthly )
alanf
phpLister
 
Posts: 6
Joined: 11:14am, Tue 14 Jul, 2009

Re: v2.10.10 New daily RSS users get the whole of the database

Postby H2B2 » 2:05am, Fri 28 Aug, 2009

I wonder whether this could be related to an issue reported by tipichris with respect to RSS preferences not saving.

Please see the second part of the following fix:

tipichris wrote:Try the mantis bug report again:

http://mantis.phplist.com/view.php?id=15285

I've posted a patch there. On my system at least this is register_globals issue. Despite the fact that PHPList appears to be include code to warn you if you have register_globals on, and code to force it off, there are several places where the code appears to assume that it is on. This is one of those.

Here's the patch to spageedit.php that solved the issue for me

Code: Select all
--- lists/admin/spageedit.php.orig      Fri May 29 21:42:49 2009
+++ lists/admin/spageedit.php   Sun May 31 20:32:45 2009
@@ -54,9 +54,9 @@
   if (isset($_POST['attr_use']) && is_array($_POST['attr_use'])) {
     $cnt=0;
     while (list($att,$val) = each ($_POST['attr_use'])) {
-      $default = $attr_default[$att];
-      $order = $attr_listorder[$att];
-      $required = $attr_required[$att];
+      $default = $_POST['attr_default'][$att];
+      $order = $_POST['attr_listorder'][$att];
+      $required = $_POST['attr_required'][$att];

       Sql_Query(sprintf('insert into %s (id,name,data) values(%d,"attribute%03d","%s")',
         $tables["subscribepage_data"],$id,$att,
@@ -73,11 +73,11 @@
   }
   if (ENABLE_RSS) {
     Sql_Query(sprintf('replace into %s (id,name,data) values(%d,"rssintro","%s")',
-       $tables["subscribepage_data"],$id,$rssintro));
+       $tables["subscribepage_data"],$id,$_POST['rssintro']));
     Sql_Query(sprintf('replace into %s (id,name,data) values(%d,"rss","%s")',
-       $tables["subscribepage_data"],$id,join(',',$rss)));
+       $tables["subscribepage_data"],$id,join(',',$_POST['rss'])));
     Sql_Query(sprintf('replace into %s (id,name,data) values(%d,"rssdefault","%s")',
-       $tables["subscribepage_data"],$id,$rssdefault));
+       $tables["subscribepage_data"],$id,$_POST['rssdefault']));
   }
   if (!empty($_POST['activate'])) {
     Sql_Query(sprintf('update %s set active = 1 where id = %d',



See also viewtopic.php?p=67474#p67474
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: v2.10.10 New daily RSS users get the whole of the database

Postby metropolitan » 5:43pm, Mon 26 Oct, 2009

The scenario described above is not even the worst case. Imagine your PHPlist
pulls an RSS feed that has only a couple posts a month, so that there is a year's
worth of posts in the PHPlist database for that list. Now a user comes along and
signs up for a daily digest. I think most people would not expect a daily
digest to then send them the last year's entries. A common sense
expectation is you might get either 1) the previous day's entries ; or 2)
just new daily digests from that point on. Even a couple days' worth may not
cause most users to feel like something is wrong, but when you send them
a "dump" of stuff much of which is way out of date, they are not going
to feel like the signal to noise ratio looks good for this new list they
signed up for.

My view is that this is a bug. It has been reported by multiple
PHPlist users as a problem (see viewtopic.php?f=15&t=24458#p67778)

I've applied the 15285 patch and it did not solve the problem -- shouldn't this get
its own bug report? So I posted it at http://mantis.phplist.com/view.php?id=15355

I have tested a solution to this and the RSS entries are out-of-order bug.
There are two facets to this fix. This first (in getrss.php) involves setting
the rssitem "added" field to have a datetime value based on the pubdate
from the RSS entry rather than now(). The pubdate is available from the
RSS entry details in the database. As a result of this change, RSS digest
entries are maintained in appropriate chronological order (resolving
RSS bug Mantis #14003). The second facet is to make a change to the
rssUserHasContent() function in rsslib.php that computes the age of an RSS
entry and compares that with the digest interval. If the RSS item is older,
then it doesn't push it onto the array of items to send. This results in the
digests not having items older than the digest interval. The coding needs
to be made more robust in order to handle edge cases but with these
two bugs quashed, RSS digests become more user-friendly.

erik

ps--I'm not using the Simplepie RSS fetcher.
metropolitan
phpList newbie
 
Posts: 1
Joined: 5:21pm, Mon 26 Oct, 2009

Re: v2.10.10 New daily RSS users get the whole of the database

Postby deancanham » 9:50pm, Wed 06 Jan, 2010

The RSS feed is the most important tool from my point of view. I am using a blog entry as a regular newsletter feed. As with everyone else, I am having issues with new subscribers getting all blog entries, going back over a year. This is not acceptable from a subscribers point of view. I am surprised that more people are not concerned about this. What Erik has said is all correct - it just needs someone with the coding know-how to do it. Please help!!! (if it helps, I will make a donation to PHPList if this is fixed, otherwise I will be considering some other system)

Thanks,

Dean
(ps. using simpliepie due to an atom feed)
deancanham
phpLister
 
Posts: 5
Joined: 11:48am, Fri 04 Sep, 2009

Re: v2.10.10 New daily RSS users get the whole of the database

Postby Mhartvig » 12:27am, Fri 18 Feb, 2011

Think i solved this problem by a simple fix. (update the SQL row with the correct pubdate after it is inserted)

(Dirty hack - do it at your own risk)
** I havent tested mailings yet, but "pubdate" gets the correct value now in the database - Works on my setup, but i also purge rss (everything older than 1 day) before running a processque cronjob - AKA it can NEVER EVER send a message older than 1 day.**

--------------
Open getrss.php from the admin folder.

Around line 111, add the following:
Code: Select all
         // Pubdate HACK
         if ($key == 'pubdate') {
            $thepubdate = strtotime(addslashes($val));
            $thepubdate = date("Y-m-d H:i:s", $thepubdate);
           Sql_Query(sprintf('update %s set added = "%s" where id = "%s"',
              $tables["rssitem"],$thepubdate,$itemid));
         }


Just to clarify, add it just above this code (dont add this anywhere, its already there):
Code: Select all
if ($item != 'title' && $item != 'link') {


WHEN YOU ARE DONE, it should look like (around line 111):
Code: Select all
foreach ($item as $key => $val) {
         // Pubdate HACK
         if ($key == 'pubdate') {
            $thepubdate = strtotime(addslashes($val));
            $thepubdate = date("Y-m-d H:i:s", $thepubdate);
           Sql_Query(sprintf('update %s set added = "%s" where id = "%s"',
              $tables["rssitem"],$thepubdate,$itemid));
         }
           if ($item != 'title' && $item != 'link') {
            Sql_Query(sprintf('insert into %s (itemid,tag,data)
              values("%s","%s","%s")',
              $tables["rssitem_data"],$itemid,$key,addslashes($val)));
            }
        }
Mhartvig
phpList newbie
 
Posts: 1
Joined: 12:07am, Fri 18 Feb, 2011


Return to Bug Discussion

Who is online

Users browsing this forum: No registered users and 3 guests