CSE 40771 - Distributed Systems - Spring 2026
This assignment will introduce a few key features needed to realize our file sharing protocol. First you will modify the client and server to discover each other via an online naming service. Next you will improve the robustness of the client to deal with common failures. Finally you will add another message type which informs the client of what data the server has to offer.
To this point, your server has listened on a manually-selected port number in a fixed location. This is ok for testing purposes, but becomes a problem when running multiple servers across many machines. The client needs a better way of locating the server that it wants to access.
We will address this problem by making use of a simple name server that we have running here at Notre Dame: take a look at catalog.cse.nd.edu:9097. Here is how the name server works:
Various services running at ND (and around the world) periodically register
themselves with the name server by sending a UDP packet to catalog.cse.nd.edu:9097,
typically once every five minutes. The UDP packet contains a JSON document
that describes the essential properties of the service: name, port, location, memory, disk, etc.
Some of the services are quite simple, while others are very complex.
The name server publishes the set of known services via a web page. You can browse an HTML representation of the web page manually, or you can access a JSON representation programmatically, like this:
curl http://catalog.cse.nd.edu:9097/query.json | json_pp
The name server periodically discards records from services that have not sent an update in the last 15 minutes. This is a garbage collection measure to ensure that records don’t accumulate forever. So, servers must periodically refresh their state, and clients must accept the fact that any data in the name server is necessarily “stale”.
For this assignment, you must modify your client and server to use the name server as follows:
catalog.cse.nd.edu:9097. The type field
should be hashtable, the port field should be the port the server is listening on,
the owner field should be your netid, and the project field should be a personalized
server name that you pass in on the command line.
(Note that the name field is automatically filled in with the host name by the name server.)
As an example:{
"type" : "hashtable",
"owner" : "YOURNETID",
"port" : 1234,
"project" : "YOURNETID-A5-test5"
}
type=="hashtable" and project equal to your
server name. Then, connect to the indicated host and port number.Modify your client and server to make use of a project name on the command line. Your server should be started like this:
python HashTableServer.py YOURNETID-A5
And then start your client using the same project name:
python TestBasic.py YOURNETID-A5
Your client and server will now be able to find each other, no matter where they are located.
Take Care: To avoid collisions between students, please make sure to use a project name that contains your netid. (Honor system.)
Additionally: This will result in you modifying your HashTableClient class constructor to take a project name instead of host/port. To facilitate P2P later we will still need to connect to host/port combinations, so do not remove the existing capability. Simply add the ability to either connect by name or by host/port through arguments, @classmethod constructors, or by some other mechanism.
At this point your client has done the straightforward thing of connecting, sending a request, and waiting for a result. However, there are a variety of things that may go wrong in this sequence: the network could be interrupted, the server could crash, or (even worse) the server might get stuck and not send any response, causing the client to wait forever.
Modify HashTableClient so that if any of these undesired events happen:
then HashTableClient should print a short message, take a pause, reconnect, and try again. Note that the pause is important: if the client action fails quickly, you shouldn’t flood the network with rapid retries. A good policy is exponential backoff: wait one second after the first failed attempt, then two seconds, four seconds, etc. until success is achieved.
(Now you see why the operations must be idempotent!)
From the caller’s perspective, these retries should be completely invisible. It should look like the single function call just took a little longer before completing correctly. You are still free to print debug/info messages to show that your client is attempting to reconnect. In fact we are asking you to show this in your report.
Note: The retry functionality is only practical for clients connected via the name service. If we connected by host/port, there is not much use in trying the same address again, since a server will likely be using a different port if it went down and came back up.
Once you have this working, test it by starting, stopping, and killing the client and server in arbitrary combinations. Whatever happens, you should observe that the client always (after a brief delay) succeeds in completing its operations, and never returns a failure to the caller. Then move on to the next step:
In this assignment, instead of the regular testing format we are going to have one test called Test.py. This is going to be the foundation of the Peer which will soon envelop both the client and server functionality.
Test.py will invoke methods from HashTableClient just like previous tests, however you will need to make a couple modifications to the client and server. If we think about a file sharing protocol, we are going to need a way to tell a new client what data there is to download. In BitTorrent there is generally a server called the “Tracker” which tells new clients the name of each file being distributed, as well as a list of peers in the system.
For now we are still only concerned with two nodes, so your Test.py does not need to react to the list of peers.
get_description() -> (['image_1.png', 'image_2.png', ... ], [])
Add a message called get_description to the client api, and a handler for the message at the server. The response to this message will need to have two components which may be contained in a tuple, json or python object. The list of files should be the equivalent to calling dict.keys() on the server’s hash table. The list of peers should be a list of (host:port) combinations for each connected peer (which is only one this week). For now it should contain the (host:port) of the client who sent get_description.
When Test.py invokes HashTableClient and connects to the server, the first thing it should do is send a get_description message.
Your script should then iterate through the keys given by the server, performing a lookup on the hash table for each file until it gets all of the data.
Each time your client performs a lookup, it should write the file to disk.
Your tests will consist of the functionality described in the previous section for Test.py. Using one of your old client scripts on the same machine as the server you can insert a set of files into the hash table before you start your main test on a different machine, so it has something to download. In total make the sum of the data around 5-10MB.
You will need to kill and restart the server in the middle of the client download sequence. You are welcome to slow down the test by inserting time delays between lookups so you have time to restart the server and observe the behavior.
Measure the time it takes to complete each lookup (not including any artificial delay).
The test program should briefly pause, reconnect, and keep going without losing any results.
Please review the general instructions for submitting assignments.
Turn in all of your source code, along with a lab report titled REPORT that describes the following in detail: