Thanks for the information. We will take advantage of that and make a release tomorrow so we will deploy more stable version of Neptune. I will write information here as soon as it is done.
Thanks. I will probably not have much time to try this out this week, so take your time to make it stable
Thanks for your patience. We will let you know when the release is ready.
The release is ready. To run your code, you need to:
- upgrade your CLI to the newest version:
pip install --upgrade neptune-cli,
- change line 113 in utils/config.py to
args, _ = parser.parse_known_args()to make your script accept Neptune’s internal arguments that are added to
- set the output directory to a path under
/output(as documented in https://docs.neptune.ml/cli/neptune_send/#using-storage).
The full command to run your code is:
neptune send "utils/run_experiment.py --log /output/results" --worker gcp-gpu-medium --environment pytorch-0.1.12-gpu-py3
Please let us know how it worked for you.
Now you’re able to run your code without modifying the utils/config.py file. Neptune doesn’t add arguments to
sys.argv anymore. You just need to update the CLI to the newest version (2.0.21).
Thanks! It starts running really fast, and the UI is very responsive. I did notice that installing the cli interferes with versions of some packages and messes up my other CLIs, but I re-installed and now it seems fine. The
run_experiment.py ran fine, but I had an error while creating a directory when I ran another job, which is the real thing I’m interested in running.
neptune send "utils/select_experiment.py --config 109078-182929-with-sample-samp1.0-pc --quick --log /output/results" --worker gcp-gpu-medium --environment pytorch-0.1.12-gpu-py3
1 Traceback (most recent call last): 2 File "/usr/local/lib/python3.5/dist-packages/deepsense/neptune/job_wrapper.py", line 108, in <module> 3 execute() 4 File "/usr/local/lib/python3.5/dist-packages/deepsense/neptune/job_wrapper.py", line 104, in execute 5 execfile(job_filepath, job_globals) 6 File "/usr/local/lib/python3.5/dist-packages/past/builtins/misc.py", line 82, in execfile 7 exec_(code, myglobals, mylocals) 8 File "/source/utils/select_experiment.py", line 14, in <module> 9 conf_new, best_metrics = run(conf) 10 File "/source/utils/experiment.py", line 408, in run 11 save_artifact(params, "model_params", conf.save_dir) 12 File "/source/utils/experiment.py", line 179, in save_artifact 13 make_file_dir(obj_file) 14 File "/source/utils/experiment.py", line 221, in make_file_dir 15 os.makedirs(os.path.dirname(path)) 16 File "/usr/lib/python3.5/os.py", line 231, in makedirs 17 makedirs(head, mode, exist_ok) 18 File "/usr/lib/python3.5/os.py", line 231, in makedirs 19 makedirs(head, mode, exist_ok) 20 File "/usr/lib/python3.5/os.py", line 231, in makedirs 21 makedirs(head, mode, exist_ok) 22 File "/usr/lib/python3.5/os.py", line 241, in makedirs 23 mkdir(name, mode) 24 OSError: [Errno 30] Read-only file system: 'results'
FYI I am on travel on west coast and won’t have much time to play around until next Tuesday, but it’s looking really promising!
Also I see that Browse files needs many strange steps – like clicking “…” two times just to see my source code, and then when I click on a source file, I have to download it. This needs to be much smoother, and code should be viewable in the browser, not requiring download.
Actually I fixed that error ( I need to restrict my code to only write under
/output directory) and now it’s all running smoothly, and it’s really fast! Next week hopefully we will explore charts etc
I am glad that you managed to run your experiment. I look forward to getting feedback from you next week.
Thank you for sharing your problem with “Browse files”, we know about the issue and we work on this. Regarding output, you are right: to write on persistent storage you have to use
/output directory, the rest are read only.