Launched: Track, Organize, Collaborate - Meet the new, updated Neptune

Hello folks! :wave:

We have released a new version that is even more lightweight and easy to work with.

Have a look at our new webpage, that explains the direction our product is taking.

Neptune is focusing on 3 pillars:

  1. Track: Neptune is a vault that stores all the artifacts relevant to a data science project: input data, visualizations, notebooks, models, results and so on.

  2. Organize: Neptune provides building blocks for creating a complete knowledge repository by structuring and indexing everything you track.

  3. Collaborate: Finally, every piece of knowledge organized in Neptune can be shared, discussed and reviewed - in the privacy of your team or at a public forum.

What are the most significant changes?

By talking to our users and Data Science community at large, we realized that what people really need is flexibility and easy integration. We listened, and made Neptune extremely lightweight and easy to run everywhere. What it means is that:

  • You are no longer forced to use CLI to run your experiments. You can include Neptune in any Python code (scripts, Jupyter notebooks, etc) and run it anywhere - be it your favourite cloud or your laptop!

  • We chose to no longer provide infrastructure. In our research we discovered that users rarely need a managed infrastructure solution - instead, they asked us for something that fits in with any existing setup. In short: you choose where to run and how to run it - Neptune will fit.

  • We’ve vastly extended our Python API. You can now define experiments, log any type of data, analyze historical experiments, run hyper-parameter optimizations, etc - right in your code. You can now even treat your exploratory analysis as an experiment and log that to Neptune!

What does it mean for you?

You will receive an email with detailed information about how that change affects your account (if it affects it at all).

Please be informed that all the accounts created after 08.03.2019 are already using the newest version.

Old projects will not be automatically transferred to the new product version. If you would like to have your projects and experiments transferred to the new UI please send us an email to contact@neptune.ml.

If you have not yet switched to the new version I would like to let you know that the previous version will be maintained until 31.07.2019. In case you will not update your account it will be automatically updated to the new version as of 01.07.2019.

Read more about tracking, organization and collaboration in Neptune on our blog and in the new documentation.

Wondering how to start working with the new Neptune?

Neptune has now more tracking options, all simply available from the python code. Check out Tutorial in the nutshell.


Stay tuned for more news about the new features. They are coming soon!!

Meanwhile, feel free to contact me with any trouble related to the product update or any other questions you might have.

As always, this time more than ever, I’m very curious what you think about the changes we introduced to Neptune!

Cheers!
Paulina

Hi Paulina,

The managed infrastructure part was what I most liked about Neptune… so this is really unfortunate :frowning: Can you recommend any other service that provides that part even if I continue to use Neptune for the experiment tracking part (which is also awesome by the way)?

Thanks,

Sofia

1 Like

Hi Sofia,

Thank you so much for asking.
I am actually planning to write a blog post where I would compare/run Neptune on different platforms.
The places I am planning on checking out are AWS Sagemaker, Google Cloud ML Engine and Paperspace.

I think it is important to note, that with Neptune dropping infrastructure comes really lightweight integration. Because of that, you can run it pretty much everywhere and it should work. Some notable examples are google colab or even kaggle kernels.

I will let you know once I have the blog post ready but in all cases, we will have a good solution before we disable infrastructure.

If you try any cloud provider on your own, please share comments here. I would really appreciate it.

Thanks again,
Jakub

I second Sofia.

There are more software to track results, if needed. What I really liked about Neptune (both from my personal project and trainings) was the seamless way to run experiments in the cloud (with batteries included: tracking, saving data, comparing experiments).

When someone asked me how to run experiments, I recommended them Neptune. Now a few people asked me, and honestly - I don’t know what to recommend them (I am looking for a solution for myself as well). Sure, there is plug&play Google Colab, but it is not good enough for comparing other experiments.

Sure, now Neptune gets more lightweight. But again, the biggest added value was not needing to think about the infrastructure (as many researchers for me infrastructure is the necessary pain/burden, I avoid whenever possible).

Maybe there is some way to have instructions how to do so (not like 8 steps or so, but a built-in script)?

Hi @pmigdal,

Nice to see you here :slight_smile:

As Paulina mentioned in her post, our market research strongly suggests that it is a very rare case that someone would need managed compute infrastructure. Because of that, we have decided to go lightweight and - as a result - open Neptune to broader data science audience.

But do not worry. There is hope! In our team, @jakub_czakon is working on a short blog post that will describe, how to easily run experiments on EC2 instances on AWS.

Stay tuned :wink: .

Best,
Kamil

Hi @medula and @pmigdal,
I have some good news for you.

We have added how-to’s for running Neptune in the cloud to our docs.
You can see how to:

I hope those instructions would be useful to you.
Any feedback is highly appreciated!

Thank you, though it does not solve a problem.
I know how to set up an AWS instance :slight_smile:
But there is a lot of time involved in doing it per experiment.

Still, I don’t know is how to do it as a one-liner (or anything close to it).