SG memory usage areas for followup - adamcfraser/cbnotes GitHub Wiki

Initial thoughts on SG memory usage - items that can potentially grow large:

Change Cache

changeCache has

• channelCaches map[string]*channelCache. One entry per channel.
• receivedSeqs map[uint64]struct{}. One entry per sequence, never purged. Entries are empty structs (just getting used to identify whether a seq has already been seen)
• pendingLogs []*LogEntry. Out-of-sequence docs that haven't are waiting for processing.

Channel caches

• array of 50-500 *LogEntry
• each LogEntry has:

  Sequence     uint64
  DocID        string
  RevID        string
  Flags        uint8
  TimeSaved    time.Time
  TimeReceived time.Time
  Channels     ChannelMap

Here ChannelMap is a map of channel names (string) to a struct containing:

  Seq     uint64
  RevID   string
  Deleted bool

Not memory related, but channel cache iterates through the entire cache on append to find/remove any duplicate DocIds.

Revision Cache

We do a put to the revision cache every time a doc gets saved. Set to capacity 5000. Cache is a lruList of revisions, and a map of pointers to entries on that list.

Change Listener

Maintains a map of counters, one entry per channel - used to notify when a channel cache has been updated.

Changes processing

During continuous changes processing, the following is kept in memory per user:

During each iteration of the changes loop, the following are re-initialized. Could investigate whether the create/release of these in each loop iteration has significant GC implications

• feeds - array of channels of ChangeEntry
• names - array of string
• current - array of ChangeEntry
• various strings, sequence numbers, etc.