Homework answers / question archive / Introduction Background: 1

Introduction Background: 1

Computer Science

Share With

---

Introduction

Background:

1. Build a simple distributed file system. Submit your project write-up and code. And give a demo presentation in the class. This is due the day after the mid-term.

2. Build a defensive distributed file system (add security functionality to your distributed system). At the end of this semester, submit another the write up and code for your second project. A demo presentation 1s also required.

This project requires one person in your group to have taken an undergraduate operating system class and some experience with a distributed file system.

You will work in groups of 3—4. You are required to turn in both your code and a short write-up describing the design and implementation of your project, and to make a short in-class demo presentation about your work. Each group only needs to submit one copy of the project report and code. At the end of the semester, we will ask group members to evaluate their team members' performance. We want each group member to have equal contributions to the project. Usually, each member will get the same grade for a project.

Deliverables

There are four concrete steps to the final project, as follows.

1. Form a Group - I assigned groups via the groups function in Blackboard.

2. Project Proposal

By the proposal deadline of 7 February, you must submit a one-to-two-page proposal describing:

• Your group members list
• How you plan to address the problem: What are your assumptions of your file system and threat model?
• Tell us what you are proposing to specifically design and implement. This should include enough detail, so we know you are on the right track.

Note: If there is a change in the project that was not covered in the proposal, this must be approved by the TAs (Ran Liu and Rohit Mokashi) and Professor Tompkins. This is in no way to limit your creativity, but to make sure the project can be completed on time. Your proposal can be updated, but we need a justification.

3. Project Presentation

Prepare a short in-class demo presentation about the work that you have done for yourfinal project. The presentation will be held online. We will ask 3-4 questions related to your project. Your score will be calculated based on your presentation.

4. Write-up and Code

Write a document describing the design and implementation of your project and turn it inalong with your project's code by the final deadline. The document should be about 2-3 pages of text and possibly diagrams that helps us understand what problem you solved, and what your code does. The code and write ups will be posted online after the end of the semester. Take a look at the list of write ups from past years (2012):

http://css.csail.mit.edu/6.858/2012/projects.html and 2013:

http://css.csail.mit.edu/6.858/2013/projects.html) to get a sense of what this writeup should look like.

Projects Description:

First Project: Distributed File System

The first project is to build a general purpose distributed file system. YOU CAN NOT USE ANY EXISTING FILE SYSTEMS OR YOUR PREVIOUS PROJECTS. You need to implement this file system as a team without any external libraries or frameworks. However, some libraries are necessary to implement updates to replicas. So please consult with us before you use any external libraries. In this project, you may findJava is more suitable for your task. JMI is the only external library you need to implement replica updates. You need to define your distribution file system functionality on your own. Some useful file system examples can be found:

• Google file system: https://research.google/pubs/pub5 1/
• The Hadoop Distributed File System: https://storageconference.us/2010/Papers/MSST/Shvachko.pdf

Ata minimum, your file system should meet the following requirements:

• Users can create, delete, read, write, rename files.
• The file system should support directories, much like the Unix file system.
• Your file system should support many users and allow users to share files with one another. For each file, 1t should be possible to control the set of users who can read, and who can write, to that file.
• Each client will have a primary server that handles most of the operations and data will be replicated on multiple replica servers.
• Your file system needs to allow concurrent users to perform operations while guaranteeing ACID property.
• Your system needs to have the ability to handle 10000 requests with a maximum of 10000 requests running concurrently.

Second Project: Encrypted File System

Based on your first project, your goal for the second project idea is to allow users to store data on an untrusted file server. The file server should not be able to obtain the user's plaintext data (1.e., your file system should encrypt the data), and the file server should not be able to corrupt the data either (1.e., your file system should authenticate the data it gets back from the file server).

At a minimum, your file system should meet the following requirements:

• Users can create, delete, read, write, rename files.
• The file system should support directories, much like the Unix file system.
• Users should be able to set permissions on files and directories, which also requires that your file system be able to name users.
• File names (and directory names) should be treated as confidential.
• Users should not be able to modify files or directories without being detected unless they are authorized to do so.
• If the server is not malicious, unauthorized users should not be able to corrupt the file system.
• The file server should not be able to read file content, file names, or directory names.
• The server should not be able to take data from one file and supply it in response to a client reading a different file.
• A malicious file server should not be able to create or delete files or directories without being detected.

The result of these projects should be a functional file system implementation that meets the above requirements. You can implement your prototype in any language you want, such as Python or C++ or Go. You can decide how the file system client and server should be run. One reasonable design would be to have the file system client provide a minimal shell environment that allows users to perform the operations described in the above requirements.

As a challenge, you may want to think about providing freshness guarantee: that is, that a client should always see the latest version of a file, or at least that a client should never see an older version of a file after 1t sees a newer one. The SUNDR file system (PDF) may provide some inspiration for this, although it 1s a rather complicated system.

You can choose to implement some limited freshness guarantees, unlike SUNDR's ideal fork consistency.