Add `send_parameters()`

Currently one can only specify the parameters when a neptune experiment is created. I would like to separate the neptune experiment creation from setting the parameters. I suggest to add send_parameters(dict_).

Hi @sotte,

Nice to hear from you here :slight_smile:
As we envisioned expriments in Neptune is that, parameters are an integral part of an experiment that you pass once (as dict) during experiment creation.

Would you like to tell me more about why you’d like to have those two separated?

Best,
Kamil

Hi @kamil.kaczmarek, it’s mostly a code organization thing. I have a more generic infrastructure layer and something that is more specific to the training. In my case it makes sense to create the neptune experiment in the infrastructure layer, but I don’t have the config parameters there yet. I want to keep the training part as focused on the training as possible and I don’t want to put neptune.init()... into the training part.

Offering a send_parameters() also imposes less workflow restrictions than not having that function.

Hi @sotte,

We designed it in this way to make sure that User will not accidentally change parameters during experiment execution -> it may have detrimental impact on the reproducibility.

I have two ideas how it might work

Idea 1

In general there are two ways to work with Neptune projects and experiments. The one you mentioned:

neptune.init('org/project_name')
neptune.create_experiment(params=my_dict_params)
# training code follows

And another, like this:

# create Neptune session and get project
project = neptune.Session().get_project('org/project_name')

# create experiment
npt_exp = project.create_experiment(params=my_dict_params)

You can create project in the infrastructure layer, and experiment in the training layer.

Idea 2

You can use neptune.set_property('key', value) to keep track of your parameters that you compute during experiment. In such case you do not need parametes at all. In the application, properties are placed right below parameters, in the details tab. See example.

Best,
Kamil

Hey,

I would argue that people still can change parameters during experiment execution even when you have to specify the params for create_experiment(). I actually have to do that right now, because I add the project id (and a derived path for the project) to the config. Something along these lines:

neptune.init('org/project_name')
exp = neptune.create_experiment(params=my_dict_params)
my_dict_params["project_id"] = exp.id
my_dict_params["artifact_dir"] = "some/path/{exp.id}"
Path(my_dict_params["artifact_dir"]).mkdir(parents=True)

I guess I could use properties. Still, I would prefer the flexibility of being able to add params whenever I want.

Hi @sotte,

Thanks for that.

In the snippet provided, you are adding more key-value pairs to the my_dict_params. This sounds reasonable to me, that is to append key-value pairs to params during experiment execution. Would that feature satisfy your needs?

I still think it is important - for the reproducibility purposes - not to overwrite params or change selected entries during experiment.

@jakub_czakon what do you think about this idea?

Best,
Kamil

The additional params I add to my_dict_params are not tracked with Neptune though.

You could offer something like neptune.config that only allows to add stuff. Then you don’t have any problems with reproducibility.