Workflow with an evolving codebase


When performing experiments, I might still make changes to the source code. Hence, I find myself removing old versions and reuploading slightly modified versions of input code/data to Neptune.

This feels bad for two reasons:

  • I have to keep track of modified files and manually remove old and upload updated files to my project
  • I ruin the reproducibility of older experiments within a project

I suppose that I can put everything that might change into a notebook or the main script, however, this does not seem like a perfect solution.

What is the recommended way to work with an evolving codebase?


the answer to your question depends on the setup that you are running.

For example if you are developing code on your local machine and simply execute it via neptune by running either neptune run or neptune send ... you can (and i would advice you to) use version control like git with no problems.

I will assume that you are working in the neptune cloud notebooks with some files/folders that you treat as packages and those files/folders are being uploaded/downloaded frequently.
I have to say that I personally don’t use this setup very often but here are the things that I think help a lot:

Using cells as bash with:
! cp -rf /input/project_codebase /output/project_codebase to copy my codebase to a folder with write access if needed.

Read write files into cells:
%load magic to load a python file into a jupyter cell and be able to edit it.
%%writefile to write that content back to the file.

That way you can deal with all the adhoc changes from neptune.

Summing up for larger projects I choose to develop them locally in IDE and use neptune to run/monitor/manager experiment artifacts.
For smaller add hoc ideas/projects I use neptune notebooks (then I don’t really have evolving codebase). Using notebook magics described before helps a lot.

That being said we are working on improving user experience in this particular domain (and others) and there are some exciting changes coming down the pike so stay tuned!