[Fab-user] Automating remote interactive shell

[Claudius Heine]

Hi everyone,

I am currently looking for something that automates some embedded Linux
development steps on a remote build server and fabric might just be the
right tool for this.

But there is one issue where I would like your thoughts on. One part of
developing embedded Linux systems is connecting to the system via a
serial terminal in order to input some commands and read their output.

AFAIK their is currently no real solution to automate interactive
programs in fabric. There is the 'open_shell' command, but that just
gives the control to the user.

I could imagine that this functionality could be implemented in fabric
using a context manager. This implementation should then wrap around
_run_command and _execute in order to provide a compatible experience.
And the context manager provides the necessary settings (shell command,
shell prompt, shell exit sequence, ... and how the exit codes of the
programs can queried, etc.). There should also be a much simpler API
similar to pexpect that is used by the _run_command and _execute wrapper
for these higher functionality.

Is such functionality on the roadmap or do you know about something that
already exists outside of fabric that solves these issues?

Thanks and have a nice day,
Claudius

[Jeff Forcier]

Hi Claudius,

First, there is a very basic responder functionality currently implemented,
the 'env.prompts' setting:

http://docs.fabfile.org/en/1.13/usage/env.html#prompts

If I had to guess, I'd guess it's too simple for a "fully automatic"
interaction with a serial console, but it's worth trying if you haven't
already.

Second, stronger expect-like behavior is on the roadmap; version 2 (alpha
out very soon!) is built on the Invoke library, which has its own "2.0"
rewrite of the above feature, as 'watchers':

http://docs.pyinvoke.org/en/latest/concepts/watchers.html

http://docs.pyinvoke.org/en/latest/api/watchers.html#invoke.watchers.StreamWatcher

I haven't needed anything more complex yet than the sudo-oriented
FailingResponder yet, but as you can see from the generic StreamWatcher
class, the intent is to allow for pretty much any "examine output streams,
yield responses" use case (and for users to write their own instead of
being limited to what's in core; though of course any generally useful
patches are likely to be accepted.)

The specifics may yet change (e.g. attempting to submit deltas of the
output streams instead of the full captured text) but the overall concept
of stateful per-thread watcher instances seems likely to stick.

Best,
Jeff

Post by Claudius Heine
Hi everyone,
I am currently looking for something that automates some embedded Linux
development steps on a remote build server and fabric might just be the
right tool for this.
But there is one issue where I would like your thoughts on. One part of
developing embedded Linux systems is connecting to the system via a
serial terminal in order to input some commands and read their output.
AFAIK their is currently no real solution to automate interactive
programs in fabric. There is the 'open_shell' command, but that just
gives the control to the user.
I could imagine that this functionality could be implemented in fabric
using a context manager. This implementation should then wrap around
_run_command and _execute in order to provide a compatible experience.
And the context manager provides the necessary settings (shell command,
shell prompt, shell exit sequence, ... and how the exit codes of the
programs can queried, etc.). There should also be a much simpler API
similar to pexpect that is used by the _run_command and _execute wrapper
for these higher functionality.
Is such functionality on the roadmap or do you know about something that
already exists outside of fabric that solves these issues?
Thanks and have a nice day,
Claudius
--
DENX Software Engineering GmbH, Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
PGP key: 6FF2 E59F 00C6 BC28 31D8 64C1 1173 CB19 9808 B153
Keyserver: hkp://pool.sks-keyservers.net
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user

[Claudius Heine]

Hi Jeff,

Post by Jeff Forcier
Hi Claudius,
First, there is a very basic responder functionality currently
    http://docs.fabfile.org/en/1.13/usage/env.html#prompts
If I had to guess, I'd guess it's too simple for a "fully automatic"
interaction with a serial console, but it's worth trying if you
haven't already.

Hmm... Yes, I think its a bit limiting when you only have a 1:1 mapping
for prompt to input. It would be great when those sub-shells are
treated the same as the the rest of the program. So if something like
"run" or "sudo" is available in this shell, then use it, otherwise just
use something like pexpect. I would try to implement something like
shell capabilities that can be set in the shell-context. Maybe even
being able to automate the input to a python shell on the remote target
;) Not sure how useful that would be tough.

Post by Jeff Forcier
Second, stronger expect-like behavior is on the roadmap; version 2
(alpha out very soon!) is built on the Invoke library, which has its
    http://docs.pyinvoke.org/en/latest/concepts/watchers.html
    http://docs.pyinvoke.org/en/latest/api/watchers.html#invoke.watch
ers.StreamWatcher
I haven't needed anything more complex yet than the sudo-oriented
FailingResponder yet, but as you can see from the generic
StreamWatcher class, the intent is to allow for pretty much any
"examine output streams, yield responses" use case (and for users to
write their own instead of being limited to what's in core; though of
course any generally useful patches are likely to be accepted.)

Oh great, I will take a look at it.

I think that could be done generic enough to be useful for upstream.

Maybe its possible to have a class with the invoke.context.Context
methods + pexpect style ones that uses the builder pattern or similar
to create a watcher that then responds to the program output.

Cheers,
Claudius