quicknotes/specs/feeds.md

Feeds Specification

Overview

The Feeds module allows users to subscribe to and read RSS/Atom feeds. It provides feed management, automatic fetching of new entries, and a clean reading experience for feed content.

Data Model

Feed

The Feed entity has the following attributes:

Field	Type	Description
ID	string	Unique identifier for the feed (UUID)
Title	string	Title of the feed
URL	string	URL of the feed (RSS/Atom)
Description	string	Description of the feed
SiteURL	string	URL of the website associated with the feed
ImageURL	string	URL of the feed's image or logo
LastFetched	timestamp	When the feed was last fetched
CreatedAt	timestamp	When the feed was added
UpdatedAt	timestamp	When the feed was last updated

Entry

The Entry entity represents a single item in a feed:

Field	Type	Description
ID	string	Unique identifier for the entry (UUID)
FeedID	string	ID of the parent feed
Title	string	Title of the entry
URL	string	URL of the entry
Content	string	HTML content of the entry
Summary	string	Summary or excerpt of the entry
Author	string	Author of the entry
Published	timestamp	When the entry was published
Updated	timestamp	When the entry was last updated
ReadAt	timestamp	When the entry was marked as read (null if unread)
FullContent	string	Full content of the entry (if fetched separately)
CreatedAt	timestamp	When the entry was added to the system
UpdatedAt	timestamp	When the entry was last updated in the system

UUID Generation

Both Feed and Entry entities have GORM BeforeCreate hooks which ensure that a UUID is automatically generated if the ID is not provided:

• For the Feed entity:

  func (f *Feed) BeforeCreate(tx *gorm.DB) error {
      if f.ID == "" {
          f.ID = uuid.New().String()
      }
      return nil
  }
  

• For the Entry entity:

  func (e *Entry) BeforeCreate(tx *gorm.DB) error {
      if e.ID == "" {
          e.ID = uuid.New().String()
      }
      return nil
  }
  

Features

Feed Management

1. Subscribe to Feed: Users can subscribe to new feeds by providing a URL
2. Unsubscribe from Feed: Users can unsubscribe from feeds
3. List Feeds: Users can view a list of all subscribed feeds
4. View Feed Details: Users can view details about a specific feed

Entry Management

1. List Entries: Users can view a list of entries from all feeds or a specific feed
2. View Entry: Users can view the content of a specific entry
3. Mark as Read: Users can mark entries as read
4. Mark as Unread: Users can mark entries as unread
5. Filter Entries: Users can filter entries by read/unread status

Feed Fetching

1. Manual Refresh: Users can manually refresh feeds to fetch new entries
2. Automatic Refresh: The system can automatically refresh feeds at regular intervals
3. Content Extraction: For feeds that only provide summaries, the system can fetch the full content

API Endpoints

Method	Endpoint	Description
GET	/api/feeds	List all feeds
POST	/api/feeds	Subscribe to a new feed
GET	/api/feeds/:id	Get a specific feed by ID
PUT	/api/feeds/:id	Update a specific feed
DELETE	/api/feeds/:id	Unsubscribe from a feed
GET	/api/feeds/:id/entries	List entries for a specific feed
GET	/api/feeds/entries	List entries from all feeds
GET	/api/feeds/entries/:id	Get a specific entry by ID
PUT	/api/feeds/entries/:id/read	Mark an entry as read
PUT	/api/feeds/entries/:id/unread	Mark an entry as unread
POST	/api/feeds/refresh	Refresh all feeds
POST	/api/feeds/:id/refresh	Refresh a specific feed

Frontend Routes

Route	Description
/feeds	List of subscribed feeds
/feeds/:id	View entries from a specific feed
/feeds/entries/:id	View a specific entry

Implementation Details

Feed Parsing

The system uses the gofeed library to parse RSS and Atom feeds:

func (s *Service) parseFeed(feedURL string) (*gofeed.Feed, error) {
    fp := gofeed.NewParser()
    feed, err := fp.ParseURL(feedURL)
    if err != nil {
        return nil, fmt.Errorf("failed to parse feed: %w", err)
    }
    return feed, nil
}

Feed Refreshing

When refreshing a feed, the system:

1. Fetches the feed content from the URL
2. Parses the feed to extract entries
3. Compares entries with existing ones to identify new entries
4. Saves new entries to the database

Content Extraction

For feeds that only provide summaries, the system can optionally fetch the full content of entries:

1. Extract the URL of the entry
2. Fetch the web page at that URL
3. Use a content extraction algorithm to extract the main content
4. Save the extracted content as the entry's full content

User Interface

Feed List

• Displays a list of subscribed feeds with titles, descriptions, and icons
• Shows the number of unread entries for each feed
• Provides buttons for refreshing, editing, and unsubscribing

Entry List

• Displays a list of entries from all feeds or a specific feed
• Shows entry titles, summaries, publication dates, and read status
• Provides filters for read/unread status
• Includes buttons for marking as read/unread

Entry Viewer

• Displays the entry content in a clean, reader-friendly format
• Shows the entry title, author, and publication date
• Provides buttons for marking as read/unread and returning to the list
• Includes a link to the original article

Subscribe Form

• Input field for the feed URL
• Automatic detection of feed format (RSS/Atom)
• Preview of feed details before subscribing