Providing user control over scope of experiment tracking functionality

[ekhl]

(forking #63 (comment) into its own issue)

Context

Currently all tracked entities (metrics, parameters, artifacts, etc.) are associated with a job and a job starts with import foundations and ends with the exit hook. Though convenient in some cases, this can be restricting when a user wants to have more granular control over when a job should start and end (for example, when a user wants to define multiple jobs in a single python process, or when a user is developing interactively like in a notebook).

Proposal

Open to design ideas. We can close this and create more concrete issues after arriving at an agreed upon design
Some considerations for the design:

• maintain backwards compatibility with the existing behaviour
• embrace uniqueness of notebook user workflow (e.g. difficult to anticipate kernel restarts, context managers don't span across cells, execution order of cells is not guaranteed, etc.)
• interplay between allowing for multiple jobs in a process and the current recommended approach for hyperparameter and model searches

[ekhl]

@shazraz @mohammedri bringing the conversation about tracking functionality from #63 to a new issue.