REQUIRED: File System Organization

  • Create at least one git repository for your project (any hosting service is fine), but don’t worry about adding anything yet. If your project can be reasonably divided into core, reuseable pieces (generic analyses, e.g.) and particular application(s) (figure covering particular time period for a report, series of different experiments, e.g.), consider creating multiple repositories, for example one for the core components and others for specific applications.
  • Arrange the project in your local filesystem, and connect the pieces to relevant remote services (e.g., you might be using Dropbox to share inputs or outputs). Think about project needs: what inputs need to be shared? what outputs? what is the core software? how will this project be used on other platforms (e.g., collaborators machines, super computer)? Translate these into a directory structure, and identify the need for any local configuration files.
  • Write a README that explains the project layout within the filesystem (e.g., where inputs and outputs end up, what different parts of the source code cover). Also explain what SHOULD and SHOULD NOT be in the repository, as well as who will use the repository and how. Include the README file in the repository.

Documentation for Project Organization

  • Write a setup “script” for your project. This should just be a series of steps in pseudo-code English covering what needs to happen to make a new computer ready to work on / use the project. These steps should be comprehensible to a new collaborator on the project. Make note of areas where this is currently more complicated than you like, and could potentially be simplified.

  • Write a platforms overview for your project.
    • What operating systems does the project run on now?
    • What settings do you need it to run in?
    • Which ones do you want it to run in? E.g., your project is on Windows machine, collaborator wants to run it in Linux.
    • What system infrastructure does your project need now (e.g., Python / R interpretter, C/C++ compiler, database program, specific libraries)?
    • Which of those dependencies do you need to remove (or which new ones do you need to add)? Explain why in a sentence or two for these needs and wants.
  • Write a collaboration overview for your project.
    • How does development for your project work now? How many people can make changes to code? to input? to requirements (i.e., the needs / demands / constraints etc. you might have defined during yesterdays project work)?
    • How do those people communicate needed changes, or accomplishment of work? How does the team track these communications?
    • How many people are planned for the team in the future? What role will those future collaborators play? What is the plan for bringing them into the project?
  • Relate your answers to the previous question to your process diagrams or pseudo-code from yesterday’s project work. Which steps rely on which capabilities? Which determine organizational choices?

Tools

  • List the dependencies for your project (e.g., R libraries you use, nodejs packages, C/C++ headers)? What has to be done when they are updated? Research via the web the options you have for dependency management.

  • List the templating tools for the language(s) associated with your project. That is, tools which build code skeletons automatically.

  • The tool make and Makefiles are useful approach to documenting the relationships between various parts of your project (see this example). Try writing a makefile for the dependencies in your project

  • List programs you use to edit code (of any kind). What special features does those programs offer to help you with your coding task (compared to editing with just a plain text editor)? What do you like about these tools? What do you wish they did differently (or what more they did)?