Hi, can you clarify how long running notebook cells are treated? Specifically, I have a notebook model training cell which takes many hours. I execute the cell and I see the IPython [*] and the Kernel indicator is filled and print statements in the cell produce output in the nb. I then close the web page to launch another notebook in Neptune. I can see from the projects page that the first nb is “running”. Later I restart the first notebook and I no longer see the [*] for the cell but I do see the filled Kernel circle indicating kernel is running. I no longer get print output to the nb. Also this process dies at 3hrs at it seems that Neptune thinks the notebook is inactive (?). I tried cloning and re-running this nb and NOT closing the page. This seems to work as it has been running for 9hrs and printing to the nb. Is it really the case that one must leave a nb open on Neptune for it to work past 3hrs? How does one run many notebooks simultaneously on Neptune?
According to neptune documentation: “Each running notebook will stop automatically after 3 hours of inactivity.”
Main purpuse of using notebooks is to prototype your experiment code.
To run your experiment please try to use from command line:
neptune experiment send
This experiment will be executed on cloud and will be finalized right after your source code reach end or after exit statement.
Neptune will charge only for your sorce code execution time.
Thank you. I had seen the docs but was confused by the definition of “inactive” which I thought meant that no processes were running; i.e., that the 3hr inactive was a safeguard to inadvertently leaving an instance up. I see you changed the web GUI to say “Waiting for…” as opposed to “Running for…” which I think clarifies things for users.