Nilestore is built using Kompics framework, following the same design of Tahoe-LAFS. In this document we are describing the design of Nilestore.

Note: to understand the following sections you have to read the programming manual first.

briefly Kompics is a message-passing and component model framework for building distributed systems, it was designed to simplify the development of complex distributed systems.

There are three main node types in Nilestore; Peer, Introducer, Monitor, and Bootstrap.

It is the main node in Nilestore, it provides the functionality of the distributed storage system by allowing users to put/get files into/from the grid.

NsPeer is the main component in Nilestore, it requires Network, NetworkControl, and Timer ports and provides NilestorePeer and Web ports as shown in the next Figure below. it encapsulates different functional components that we will discuss each in details in the following sections.

NsAvailablePeers is an abstraction for the underlying overlay, it holds information about other living peers in the grid. it requires Timer, Network, and FailureDetector ports and provides AvailablePeers and APStatus ports as shown in next figure.

In Nilestore, we have two different overlay implementation centralized and decentralized. note that the main components that changes according to overlay type are NsShareFinder and NsPeerSelector because both try to solve either where to put the shares? or where to find the shares?.

• Centralized Version in this case each peer upon start announces its presence to an Introducer node as illustrated in Tahoe-LAFSBasics.

when a GetPeers(peerselectionIndex) is triggered at the AvailablePeers port, NsAvailablePeers will permutes the list of servers by Hash(peerid+peerselectionIndex) then it will triggers back a GetPeersResponse(permutedservers).

• Decentralized Version[1] in this case each peer -only who have a storage server- contribute in a DHT Chord ring. As described in the next figure, decentralized version of NsAvailablePeers has a Chord component which holds the information about the Chord Ring that we are part of and BootstrapClient which communicates with the BootstrapServer which is responsible for initializing the ring.

when a GetPeers(peerselectionIndex) is triggered at the AvailablePeers port, NsAvailablePeers will triggers a Chordlookup event at ChordSO then, after receiving ChordlookupResponse it will triggers back a GetPeersResponse(peer) where peer is the closest peer to peerselectionIndex in the ring.

In case where a peer does not have storage enabled so, NsAvailablePeers will not create a Chord component as specified before but instead it will communicate with the Bootstrap Node to get a random node located inside the ring then, it will communicate with that node directly using Network and will ask for assistance to discover nodes to be used either for peer selection or share finding.

NsConnectionFailureDetector (CFD) is responsible for detecting connection failures by keeping track of the network status "Session Open, Session Close" through NetControl port and by using Timer timeout events and PingFailureDetector component "Kompics component tweaked by adding number of tries instead of pinging forever".

CFD acts as a wrapper for PingFailureDetector component as it holds it inside to be used by the the CFD itself and provides it to other components by providing the FailureDetector port.

when NotifyonFailure event is triggered at the CFailureDetector port, CFD will schedule a timer event with a specified delay, if the timer timeout reached before receiving a CancelNotifyonFailure from the requester CFD will trigger a StartProbePeer(addr,retries) which will ask the PingFailureDetector to conclude if "addr" is alive or not by pinging for "retries" times. if a failure received then, a ConnectionFailure will be triggered on the requester otherwise the CFD will schedule another timer event and so on.

A sequence diagram of CFD operation is described in figure below. note that for clarity the Network part is not included.

NsRedundancy encapsulates the redundancy logic, it communicates with other components through Redundancy port which accepts Encode/Decode requests and delivers EncodeResp/DecodeResp responses. currently NsRedundancy implements the redundancy logic using the onion network coders "Reed Solomon Codes".

For the first time Encode/Decode event is triggered at the Redundancy port, the NsRedundancy will create an onion network PureCode[2] object according to the specified replication parameters and add that object to a map as shown in next table. after that, for all encode/decode events triggered again with the same parameters the NsRedundancy will use the existing coder instead of creating a new one.

NsStorageServer component is responsible for dealing (read/write/inquiry) with share files on disk through the usage of NsBucketWriter and NsBucketReader components which allows remote
read and write operations.

NsStorageServer requires Network and CFailureDetector ports, it accepts the following events on the Network port:

• AllocateBuckets event which holds information for share files to be allocated, and triggers back a AllocateBucketsResponse holding the addresses (if possible) of the created NsBucketWriters to communicate with them directly.
  
• GetBuckets event which asks about existing shares for a particular storage index, and triggers back a GetBucketsResponse holding the addresses of the created NSBucketReaders to communicate with them directly.
  
• HaveBuckets event which asks about existing shares for a particular storage index, and triggers back a HaveBucketsResponse holding a list of the existing share numbers.

NsStorageServer provides SSStatus port which accepts SSStatusRequest event and delivers SSStatusResponse event holding the current status of the storage server.

NsBucketReader accepts RemoteRead and Close events at the Network port and delivers RemoteReadResponse, NsBucketWriter accepts RemoteWrite and Close events at the Network port and delivers RemoteWriteResponse.

NsImmutableManager is responsible for creating/destroying uploaders and downloaders upon request at its provided Immutable port, also it carry information about current uploaders/downloaders.

NsUploader represents a single upload operation and it is initiated with the file handle of the file to be uploaded.

A sequence diagram of the main actions done during the upload process is described in next figure. Note that every status message returned from the NsWriteBucketProxy is evaluated to check BucketWriter failures but we didn't add them to the sequence diagram to make it clear.

NsPeerSelector is responsible for selecting the appropriate peers for the uploading process. NsPeerSelector is an abstraction for the peer selection process and it has different implementations, in case of using the Introducer node "centralized" then a Tahoe2PeerSelector algorithm is used otherwise in case of using Chord "decentralized" then a ChordPeerSelector[1] is used.

Despite the used type of NsPeerSelector, a NsPeerTracker components are used where each NsPeerTracker is responsible for communicating with one storage server to ask about existing shares and/or to create new shares.

In Nilestore, we have different implementations of PeerSelectors; Tahoe2PeerSelector and ChordPeerSelector[1] which used in case of Chord ring overlay . ChordPeerSelection algorithm is inspired by the proposed Tahoe3PeerSelector algorithm and it's a gossip like algorithm where we ask random peers in the ring to hold the task of allocating a share for us either on their storage server or by propagating the request to other peers and finally returning back with a peer that accepted to hold that share. this algorithm theoretically could fit very well in large networks. In a decentralized version the NsShareFinder will use almost the same algorithm but instead of allocating a share it will search for a share.

NsWriteBucketProxy is responsible for writing share data on a storage server by communicating with NsBucketWriter component in the storage server. At the end of peer selection process the uploader will create a set of NsWriteBucketProxy components to pass their ids to the NsEncoder for further use.

NsEncoder is responsible for the actual uploading process where the file is encrypted, segmented and then adding redundant data using NsRedundancy, then sent to servers through the NsWriteBucketProxies created by the parent uploader. At the end of the process the Encoder will trigger an EncoderDone event holding the verifyCap of the uploaded file.

NsDownloader is responsible for a single download operation and it's initiated with the capability uri of the file to be downloaded. it holds the NsDownloadNode component which is responsible for the actual downloading process, the NsDownloader is just a wrapper for NsDownloadNode with a decryption key to decrypt validated segments gathered by the download node.

NsDownloadNode holds a DownloadCommon object which holds the uri extension block(UEB), share hash tree, ciphertext hash tree and other parameters required to validate the shares.

Notice that in our current implementation we fetch the whole block hash tree and ciphertext hash tree which conflict with the use of merkle trees "validating random nodes on the tree" but it's implemented that way for simplicity we could improve that in later versions.

check the sequence diagram of the download process in the next figure.

NsShareFinder is responsible for locating shares on storage servers, it uses set of NsPeerTracker components to communicate with storage servers. it follows almost the same design as the share finder in Tahoe.

NsReadBucketProxy is responsible for reading share data located on a storage server by communicating with NsBucketReader component in the storage server, also responsible for validating blocks data against their block hash tree. NsDownloader creates NsReadBucketProxy components every time it got GotShares event from the share finder, and passes their ids to the NsSegementFetcher.

After initiating a NsReadBucketProxy component it will request the header data of the share file it is responsible for and any requests triggered at ReadBP port before getting the header data will be postponed until getting the header data. NsDownloader will trigger SetCommonParams event on the ReadBP port of read bucket proxies who don't have the number of segments and the size of the tail block.

When GetBlock is triggered the read bucket proxy will try to fetch the required hashes to verify that segment and will triggers back the ciphertext hashes in a GotCiphertextHashes event to the downloader which validates them according to the ciphertext hash tree in DownloadCommon.

NsSegmentFetcher is responsible for fetching segments upon request. NsDownloader will request segments from NsSegmentFetcher by triggering GetSegment on the SegmentFetcher port event. NsSegmentFetcher will triggers a GetSegmentResponse event when it got a segment. it follows almost the same design as segment fetcher in Tahoe but instead of creating a new segment fetcher for each segment we used it for all segments by tweaking the segment fetcher design.

NsWebApplication is responsible for handling web requests triggered at its Web port and figure which operation to be done; for example if operation is upload a file then NsWebApplication will trigger a Upload event at the Immutable port.

It is the entry point for new peers to discover already existing peers. It follows Publish/Subscribe pattern.

Introducer node has a NsIntroducerServer component which accepts Publish and Subscribe events through the Network port and delivers Announce event as shown in Figure below.

It is a server that collect data from peers whom enabled the monitoring feature in the grid, each peer will have a NsMonitorClient which communicates with other components inside the peer to get status then it will send a status every some period of time according to configuration to NsMonitorServer.

For our current implementation NsMonitorClient has a SSStatus port which used to get status information about storage server on each peer, all these information is sent to the NsMonitorServer which display these info in a visual representation. for now we have two visual representation; Storage Matrix which plot storage indeces against storage servers and display the count of shares as an intensity, and Storage Grouped View which plots the amount of contribution "Used Space and Count of Shares" by a storage node to the whole network.

Simulator is a node that creates a small-scale Nilestore grid in one java process, it has an interactive web interface to communicate with the grid as shown below. Kompics provides a P2pOrchestrator component which provides a Network and Timer ports and derives the execution from Experiment port.

nilestore/

|-- nilestore-availablepeers

|   |-- nilestore-availablepeers-centralized

|   |   |-- nilestore-availablepeers-centralized-cmp

|   |   `-- nilestore-availablepeers-centralized-port

|   |-- nilestore-availablepeers-decentralized

|   `-- nilestore-availablepeers-port

|-- nilestore-common

|-- nilestore-connectionfd

|   |-- nilestore-connectionfd-cmp

|   `-- nilestore-connectionfd-port

|-- nilestore-cryptography

|-- nilestore-immutable

|   |-- nilestore-immutable-abstract-updown

|   |-- nilestore-immutable-common

|   |-- nilestore-immutable-downloader

|   |   |-- nilestore-immutable-downloader-cmp

|   |   |-- nilestore-immutable-downloader-node

|   |   |   |-- nilestore-immutable-downloader-node-cmp

|   |   |   `-- nilestore-immutable-downloader-node-port

|   |   |-- nilestore-immutable-downloader-port

|   |   |-- nilestore-immutable-downloader-reader

|   |   |   |-- nilestore-immutable-downloader-reader-cmp

|   |   |   `-- nilestore-immutable-downloader-reader-port

|   |   |-- nilestore-immutable-downloader-segfetcher

|   |   |   |-- nilestore-immutable-downloader-segfetcher-cmp

|   |   |   `-- nilestore-immutable-downloader-segfetcher-port

|   |   `-- nilestore-immutable-downloader-sharefinder

|   |       |-- nilestore-immutable-downloader-sharefinder-port

|   |       `-- nilestore-immutable-downloader-sharefinder-tahoe

|   |-- nilestore-immutable-file

|   |-- nilestore-immutable-manager

|   |   |-- nilestore-immutable-manager-cmp

|   |   `-- nilestore-immutable-manager-port

|   |-- nilestore-immutable-peertracker

|   |   |-- nilestore-immutable-peertracker-cmp

|   |   `-- nilestore-immutable-peertracker-port

|   `-- nilestore-immutable-uploader

|       |-- nilestore-immutable-uploader-cmp

|       |-- nilestore-immutable-uploader-encoder

|       |   |-- nilestore-immutable-uploader-encoder-cmp

|       |   `-- nilestore-immutable-uploader-encoder-port

|       |-- nilestore-immutable-uploader-peerselector

|       |   |-- nilestore-immutable-uploader-peerselector-port

|       |   |-- nilestore-immutable-uploader-peerselector-simple

|       |   `-- nilestore-immutable-uploader-peerselector-tahoe2

|       |-- nilestore-immutable-uploader-port

|       `-- nilestore-immutable-uploader-writer

|           |-- nilestore-immutable-uploader-writer-cmp

|           `-- nilestore-immutable-uploader-writer-port

|-- nilestore-interfaces

|-- nilestore-introducer

|   |-- nilestore-introducer-port

|   `-- nilestore-introducer-server

|-- nilestore-main

|-- nilestore-monitor

|   |-- nilestore-monitor-client

|   |-- nilestore-monitor-port

|   `-- nilestore-monitor-server

|-- nilestore-peer

|   |-- nilestore-peer-cmp

|   |-- nilestore-peer-main

|   `-- nilestore-peer-port

|-- nilestore-redundancy

|   |-- nilestore-redundancy-onion-fec

|   `-- nilestore-redundancy-port

|-- nilestore-simulator

|-- nilestore-storage

|   |-- nilestore-storage-common

|   |-- nilestore-storage-immutable

|   |   |-- nilestore-storage-immutable-reader

|   |   |   |-- nilestore-storage-immutable-reader-cmp

|   |   |   `-- nilestore-storage-immutable-reader-port

|   |   |-- nilestore-storage-immutable-sharefile

|   |   `-- nilestore-storage-immutable-writer

|   |       |-- nilestore-storage-immutable-writer-cmp

|   |       `-- nilestore-storage-immutable-writer-port

|   |-- nilestore-storage-port

|   `-- nilestore-storage-server

|-- nilestore-uri

|-- nilestore-utils

|-- nilestore-web-sharedresources

|-- nilestore-webapp

`-- nilestore-webserver

    |-- nilestore-webserver-jetty

    |-- nilestore-webserver-port

    `-- nilestore-webserver-servlets