support for interactive and URI generation

[vsoch]

Interactive mode with command line interaction

jobspec nextegen should be able to accept an interactive mode, where the subinstance is expected to run at some duration or based on the parent. We could minimally output the Flux URIs for each subinstance, or even make a setup where the jobspec run will correspond with a named config file (in some determined location) so then the user can interact with the instance. For example, I might launch a jobspec with subinstances:

# Create your nested instances with jobs, etc.
jobspec run ./examples/ml-batch.yaml

# List the instances created
jobspec list-instances job://ml-batch
group-1
group-2

# Execute commands to them (wraps a flux proxy to the URI)
jobspec exec job://ml-batch:group-1 flux run echo hello world
hello world

The reason I'm thinking of having a namespace design with job:// is because we could imagine other units to execute work to. For Kubernetes it could be either of these:

# This example shows an idea to have namespaced uris for executing commands to
jobspec exec job://ml-batch kubectl exec -it <pod> bash
jobspec exec k8s://ml-batch exec -it <pod> bash

I actually think I like the first - less things to remember. That said, if it's always a "URI thing" it is a cleaner interface to do:

# This example assumes it is always a flux URI (and removes the URI prefix)
jobspec list-instances ml-batch
jobspec exec ml-batch:group-1 flux run echo hello world
jobspec exec ml-batch kubectl exec -it <pod> bash

I might like that best of all - feels very container-y :)

This would work for instances running work (non-interactive), or for an interactive instance (started with sleep), which we can do with:

version: 1
resources:
  spack:
    count: 1
    type: node

tasks:
- name: spack-builder
  interactive: true
  resources: spack

Also see related #20