RSS feeds are sent in reverse date order

Advanced Questions & Problems

RSS feeds are sent in reverse date order

Postby pnewcomer » 2:18am, Thu 23 Apr, 2009

RSS feeds are sent in reverse date order so that the oldest stories are displayed at the top of the list. Is there a way to change this so that the newest stories are displayed at the top of the list?
pnewcomer
phpList newbie
 
Posts: 3
Joined: 7:48pm, Mon 06 Apr, 2009

Re: RSS feeds are sent in reverse date order

Postby H2B2 » 4:33am, Fri 01 May, 2009

I think this query in rsslib.php, line 66 (v2.10.9) would be involved here:
Code: Select all
    $itemreq = Sql_Query("select {$tables["rssitem"]}.*
          from {$tables["rssitem"]} where {$tables["rssitem"]}.list in ($liststosend) order by added desc, list,title limit $max");
        while ($item = Sql_Fetch_Array($itemreq))


As you can see the sort order seems to be descending, so I would expect the last added rss items to be displayed on top, but I' might be wrong, since I haven't used the RSS feature. Anyway, you might try changing "desc" by "asc" and see if that produces the order you want.
H2B2
Moderator
 
Posts: 7188
Joined: 1:51am, Wed 15 Mar, 2006

Re: RSS feeds are sent in reverse date order

Postby yeahdisk » 3:49pm, Tue 16 Jun, 2009

Hi - I'm having this same problem : making the recommended change in rsslib didn't work for me.

Does anybody have a solution?
yeahdisk
phpLister
 
Posts: 7
Joined: 8:40pm, Sun 14 Jun, 2009

Re: RSS feeds are sent in reverse date order

Postby yeahdisk » 3:57pm, Tue 16 Jun, 2009

In fact, now I check it, the RSS posts are all entered into the email message in a seemingly random order.

How can this be? I'm worried that it has something to do with the way phplist imports the RSS feed in the first place (i.e. phplist doesn't read the RSSfeed in order, so when the feed is output to email it is all screwed up).

Is there a way I could improve this performance.

Oh, and I'm using the simplepie 'upgrade'.
yeahdisk
phpLister
 
Posts: 7
Joined: 8:40pm, Sun 14 Jun, 2009

Re: RSS feeds are sent in reverse date order

Postby deancanham » 12:05pm, Fri 04 Sep, 2009

I'm having the same issue, also using Simplepie. The RSS dates are the dates that I run getrss, and a it sorts by the slightly different times. Just need to substitute this with the date of the RSS, and we'll be right, but no idea how to do this.

Also on RSS issues, and I think it is related - when I try to send an RSS feed to a new member, they get all of the RSS feeds - probably because they all have a feed date that is after their join date.
deancanham
phpLister
 
Posts: 5
Joined: 11:48am, Fri 04 Sep, 2009

Old RSS feeds sent to new user

Postby deancanham » 12:25am, Mon 14 Sep, 2009

I have re-dated all of the old blog entries in the database, so that they show in the correct order. I now have a daily getrss and processqueue setup, so the date won't be such an issue - it may be a little out. The problem I still have is that if a new user joins, the first RSS email they get has all of the blog entries. Not so much of an issue now, as the blog only started last year and has less than thirty entries. But, in 5 years time, I don't want a new user to get swamped with emails. Any ideas?
deancanham
phpLister
 
Posts: 5
Joined: 11:48am, Fri 04 Sep, 2009

Re: RSS feeds are sent in reverse date order

Postby RTWolf » 5:08pm, Fri 02 Apr, 2010

I'm putting a reply here cause this gets page gets linked a lot:

Has anyone tried turning down the "Maximum amount of items to send in an RSS feed" setting in "configure"? It's set to 30 by default, which is fairly high. Something like five or ten may be more reasonable. Does that fix this issue with new users?
RTWolf
PL Nut
 
Posts: 24
Joined: 2:42pm, Wed 31 Mar, 2010

Re: RSS feeds are sent in reverse date order

Postby RTWolf » 5:29pm, Sat 03 Apr, 2010

I started working on a solution in this thread: viewtopic.php?f=24&t=31149 Ignore that partial solution.

What's below works for me, but use at your own risk. I'd love to hear feedback. Remember to comment out lines (using #) instead of deleting them so they can be put back in.

Instructions to fix arbitrary order of RSS items

Bug description: Your rss items are coming up in some arbitrary order, in viewRSS and in the emails sent out. You've likely just started to use PHPList on an ongoing RSS feed with multiple items on it.

Bug cause: RSS items are supposed to be ordered by date, of course. However, for some good reason, the programmer of the RSS functionality decided to order by the date the RSS item was added to PHPList rather than by the publication date of the original RSS item ("pubdate" in RSS feed). So, if you just put into PHPList an ongoing RSS feed that's had many items, PHPList has them all tagged with the same date added ("added" in "rssitem" table), and thus can't be ordered by it in any sensible fashion. This functionality makes sense for most RSS feeds because say you post something today and then another thing a week from now. PHPList would check today and add your new item and check again next week and add your new item. The pubdate would be mirrored by date added. But if you're adding many items at once (such as starting a new RSS-to-email functionality using an on-going RSS feed), this goes wonky cause they all have the same date added. The programmers of PHPList are doing a great job and I'm sure there's some reason I don't understand for doing this (maybe "pubdate" wasn't standardized when it was first written or whatever).

How bug fix works: This fix works by making the date an rss item is added the same as the pubdate, so when the sorting happens in viewrss and in sending out the emails, it comes out in a proper order by date posted. In emails, the order of items is from the oldest to the newest. To make this order go from newest first to oldest (which I think was what the thread starter wanted), I'll add some instructions later. My partial solution is at the bottom that doesn't work, so any help would be great. This fix only works for getrss AFTER you apply the fix. If you've already got rssitems in the database already, they will continue to come up wonky. After you apply fix, you've to purgeRSS and getRSS again.

Who this fix may not be for: If you've had an RSS newsletter going on for a while without problems, don't worry about this fix. If you're applying this to a brand new RSS feed with very few (<5) items, then don't worry about this fix. The added dates will mirror the pubdates. In both these cases, each email news letter doesn't contain more than a few items, then you may not want this fix. If, however, you're noticing that the emails that are being sent out aren't in temporal order (oldest to newest, or the opposite), then you may want to consider this fix). The reason that might be is say you post five new blog entries but only getRSS and send out emails weekly. Those five posts will have the same date added and thus will show up in an arbitrary order in the emails, instead of oldest to newest.

Who this fix is for: If each time you do getRSS you get 5+ new items (ie you post new blog posts daily but only getRSS weekly...you can just getRSS daily as an easier solution). If you are just adding RSS-to-email to an on-going RSS feed with many items. This fix will make the order of the items in emails go from the oldest to the newest and fix the order in viewrss.

Fix instructions:
Add the bolded lines towards the end of admin/getrss.php:

foreach ($item as $key => $val) {
if ($item != 'title' && $item != 'link') {
Sql_Query(sprintf('insert into %s (itemid,tag,data) values("%s","%s","%s")', $tables["rssitem_data"],$itemid,$key,addslashes($val)));
# RT Wolf's fix for arbitrary RSS item ordering - http://www.mind-manual.com
if (strtolower($key) == "pubdate"){
Sql_Query("UPDATE {$tables["rssitem"]} SET {$tables["rssitem"]}.added = '" . date("Y-m-d H:i:s", strtotime($val)) . "' WHERE {$tables["rssitem"]}.id = $itemid");
}
# RT Wolf's fix for arbitrary RSS item ordering - http://www.mind-manual.com

}
}

Old fix code:
If you're having trouble with the above code for whatever reason, try this code instead:
You have to make sure that the publication date format is correct. The way to check this is BEFORE you apply this fix, getRSS and check the viewRSS for the entries. So, when you're in viewrss, look at the publication date (not the date added) and see what the format is. So, my dates from Wordpress look like, "Sat, 03 Apr 2010 00:43:49 +0000". I made sure that the last +0000 is always zeros for me, otherwise you have to do more shenanigans if yours isn't. The format string in bold and italics below is right for this type of timestamp. Yours may differ, and if it does then you have to construct your own format string using this table: http://dev.mysql.com/doc/refman/5.1/en/ ... ate-format . I may change this to work with PHP's str_to_time later, which'll require less fiddling.
foreach ($item as $key => $val) {
if ($item != 'title' && $item != 'link') {
Sql_Query(sprintf('insert into %s (itemid,tag,data) values("%s","%s","%s")', $tables["rssitem_data"],$itemid,$key,addslashes($val)));
# RT Wolf's fix for arbitrary RSS item ordering - http://www.mind-manual.com
if (strtolower($key) == "pubdate"){
Sql_Query("UPDATE {$tables["rssitem"]} SET {$tables["rssitem"]}.added = STR_TO_DATE('" . addslashes($val) . "', '%a, %d %b %Y %H:%i:%s +0000') WHERE {$tables["rssitem"]}.id = $itemid");
}
# RT Wolf's fix for arbitrary RSS item ordering - http://www.mind-manual.com

}
}

Hope this helps! Should I put this thing somewhere else? Like mantis? Or Should it be moved to one of the other forums, like maybe as a mod/fix? I'm new so I dunno. Any help would be appreciated.

To Make Newest-to-Oldest Order of RSS Items in Emails
Near the end of admin/rsslib.php, change the bolded part to "array_unshift" without the quotation marks. array_push pushes the item to the end of the array, array_unshift pushes it to the front of the array. Doesn't seem to work, though, still sends emails in oldest to newest. Thoughts?

while ($item = Sql_Fetch_Array($itemreq)) {
Sql_Query("select * from {$tables["rssitem_user"]} where itemid = {$item["id"]} and userid = $userid");
if (!Sql_Affected_Rows()) {
array_push($itemstosend,$item["id"]);
}
}
Last edited by RTWolf on 1:46pm, Thu 08 Jul, 2010, edited 2 times in total.
RTWolf
PL Nut
 
Posts: 24
Joined: 2:42pm, Wed 31 Mar, 2010

Re: RSS feeds are sent in reverse date order

Postby kevinlee » 5:23pm, Sun 02 May, 2010

i have same problem . it sent in reverse date order. anybody can help and know to solve the problem :?
kevinlee
phpList newbie
 
Posts: 1
Joined: 5:16pm, Sun 02 May, 2010

Re: RSS feeds are sent in reverse date order

Postby deancanham » 6:46am, Tue 04 May, 2010

I was relying on this feature to automatically send out newsletters. I have not got it to work. I have since changed the way I do web pages, and now use Joomla, and am very happy with AcyMailing as a component for newsletters. Sorry I can't help.
deancanham
phpLister
 
Posts: 5
Joined: 11:48am, Fri 04 Sep, 2009

Re: RSS feeds are sent in reverse date order

Postby GazOutEast » 7:20am, Mon 27 Sep, 2010

@ RTwolf

re:
array_push($itemstosend,$item["id"]);

did you try it as

array_unshift($itemstosend,$item["pubdate"]);

The "id" is set by the sequence of fetching, and you're trying to "undo" that sequence, therefore I'd think you need a different key such as "pubdate"
If it's fixed, I can break it
GazOutEast
PL Geek
 
Posts: 61
Joined: 4:47am, Thu 08 Apr, 2010

Re: RSS feeds are sent in reverse date order

Postby GazOutEast » 10:40am, Mon 27 Sep, 2010

H2B2 wrote:I think this query in rsslib.php, line 66 (v2.10.9) would be involved here:
Code: Select all
    $itemreq = Sql_Query("select {$tables["rssitem"]}.*
          from {$tables["rssitem"]} where {$tables["rssitem"]}.list in ($liststosend) order by added desc, list,title limit $max");
        while ($item = Sql_Fetch_Array($itemreq))


As you can see the sort order seems to be descending, so I would expect the last added rss items to be displayed on top, but I' might be wrong, since I haven't used the RSS feature. Anyway, you might try changing "desc" by "asc" and see if that produces the order you want.



Again, also in this one - order by added desc - change to - order by datepub desc - otherwise all you're doing is changing the sort from asc to desc but keeping the "false" order of "date-added" to phpLIST

EDIT - Forget that ...
"pubdate" is not in table "rssitem" - it's in table "rssitem_data" and in there it's not a column it's one of the rows per item (each item has 10 rows in table "rssitem_data").
:cry: This needs a heavier fundamental rewrite of several core files to (at least) duplicate "pubdate" into a column in table "rssdata".
The whole RSS parsing system would need checked and several chunks modified to work for and with the additional data column.
:?: It's possible though, that this is the root of the sort order problem.
If it's fixed, I can break it
GazOutEast
PL Geek
 
Posts: 61
Joined: 4:47am, Thu 08 Apr, 2010


Return to Advanced Questions & Problems

Who is online

Users browsing this forum: No registered users and 2 guests

cron