I’ve added quite a few helpers to
neptune-contrib and I figured I’d compile a list of all of those here.
You can always stay up to date by checking the read-the-docs page.
Anyhow, here are the updates:
Those methods help you with versioning your datasets.
Whether it’s a file, a folder, or an s3 bucket you can log data version to Neptune.
from neptunecontrib.versioning.data import log_data_version FILEPATH='path/to/my/file' with neptune.create_experiment(): log_data_version(FILEPATH)
creates data version properties in Neptune:
log_image_dir_snapshots is a helper that lets you create a mosaic image of a sample of images from your image directory and log that to Neptune.
image_dir has subdirectories for classes it will create a mosaic for each class.
It is very simple to run:
from neptunecontrib.versioning.data import log_image_dir_snapshots PATH = '/path/to/data/my_image_dir' with neptune.create_experiment(): log_image_dir_snapshots(PATH)
and results in the following snapshot channel:
get_filepaths helps you create a list of all the files with given extensions in your project so that you can log them to Neptune.
It works like this:
from neptunecontrib.api.utils import get_filepaths neptune.create_experiments(upload_source_files=get_filepaths())
and now all your scripts are logged to
Source code section.
Check this experiment for example.
Logging matplotlib figures and pickled objects
send_figure will take your matplotlib figure and send that to Neptune.
Saves a few boilerplate lines:
from neptunecontrib.monitoring.utils import send_figure with neptune.create_experiment(): fig, ax = plt.subplots() sns.heatmap(table,ax=ax) send_figure(fig)
pickle_and_send_artifact takes your python object, pickles it, and logs this pickled file to Neptune:
from neptunecontrib.monitoring.utils import pickle_and_send_artifact with neptune.create_experiment(): rf = RandomForestClassifier() pickle_and_send_artifact(rf, 'rf.pkl')
Feedback/ideas for the future
If you like any of those, have an idea for a helper please leave a comment or drop an issue directly to neptune-contrib github page.