[FeedParser] Retrieving feeds incrementally

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[FeedParser] Retrieving feeds incrementally

Matthieu Riou
Hi,

I was checking out FeedParser and was pretty happy with its API.
However I was also looking for something that would keep on checking
the feed regularly and notify me if any new element has been posted.
Is this out of FeedParser's scope?

I'd like to implement a "server-side" aggregator that would, for
example, check a feed every 5mn and persist any new article, working
in an incremental way.

Thanks for any help,

Matthieu Riou.

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FeedParser] Retrieving feeds incrementally

Emmanouil Batsis
Matthieu Riou wrote:

>Hi,
>
>I was checking out FeedParser and was pretty happy with its API.
>However I was also looking for something that would keep on checking
>the feed regularly and notify me if any new element has been posted.
>Is this out of FeedParser's scope?
>
>I'd like to implement a "server-side" aggregator that would, for
>example, check a feed every 5mn and persist any new article, working
>in an incremental way.
>
>Thanks for any help,
>
>Matthieu Riou.
>
>---------------------------------------------------------------------
>To unsubscribe, e-mail: [hidden email]
>For additional commands, e-mail: [hidden email]
>
>  
>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FeedParser] Retrieving feeds incrementally

Emmanouil Batsis
In reply to this post by Matthieu Riou
Sorry i think i just pressed sent before :-/

Matthieu Riou wrote:

>[...] I was also looking for something that would keep on checking
>the feed regularly and notify me if any new element has been posted.
>Is this out of FeedParser's scope?
>  
>

Yes, IMHO.

>I'd like to implement a "server-side" aggregator that would, for
>example, check a feed every 5mn and persist any new article, working
>in an incremental way.
>  
>

That, depending on your needs and environment, can be implemented in a
number of ways. See about Timer, EJBTimer, Quartz Timer, etc.

hth,

MAnos

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|

Re: [FeedParser] Retrieving feeds incrementally

Matthieu Riou
Actually my problem isn't scheduling the updates, which isn't that
hard, it's more calculating the 'diff' between each check to receive
only the entries that have been updated since my last check.

The behaviour I'd like is the following (which I think is probably
what most aggregator do) :

1. Connect to a RSS feed.
2. I get all entries in the feed.
3. Connect again 5mn later.
4. I only get the entries that have been added in the last 5mn, not
the whole feed.

Of course this suppose some mechanism of 'caching' of the past feed
retrievals and also a basic diff tool that would say which entries
have been added. ROME has a submodule (Fetcher) that does something
like this but I like FeedParser's event model.

My question was whether something like this had already been
implemented in FeedParser that wasn't documented yet for example but I
guess there isn't :)

On 5/31/05, Emmanouil Batsis <[hidden email]> wrote:

> Sorry i think i just pressed sent before :-/
>
> Matthieu Riou wrote:
>
> >[...] I was also looking for something that would keep on checking
> >the feed regularly and notify me if any new element has been posted.
> >Is this out of FeedParser's scope?
> >
> >
>
> Yes, IMHO.
>
> >I'd like to implement a "server-side" aggregator that would, for
> >example, check a feed every 5mn and persist any new article, working
> >in an incremental way.
> >
> >
>
> That, depending on your needs and environment, can be implemented in a
> number of ways. See about Timer, EJBTimer, Quartz Timer, etc.
>
> hth,
>
> MAnos
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]