Discussion:
[Fab-user] fabric as library, access pool of connections indexed by host
Jon Mansey
2014-03-21 23:07:10 UTC
Permalink
Hello, my application requries opening ssh connections to a list of hosts (typically less than 10), then performing different commands on each and parsing the output. So I need a way to programatically address the ssh commands sent to the correct host. eg execute command x on host y. However I dont want to open a new connection and log in each time a command is needed, I want it to log in when first needed and keep the session open for future use.

it seems fabric's execute command should be able to do this but perhaps im just missing that there is a far simpler way of making an ssh class that can be called with a command and an index for host?

so id like to do something like this

env.hosts = [host1, host2, host3]

execute(command1, host1)

execute(commandm, hostn)


Am I on the right track here? Are there any code examples available for this use-case?

Thanks

Jon
Brandon Whaley
2014-03-21 23:12:34 UTC
Permalink
Hey Jon,

Fabric doesn't actually make multiple connections, it reuses them:

http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=cache#connections

The live connections are stored in fabric.state.connections for the life of
your program run (I believe, feel free to correct me if I'm wrong there).
Post by Jon Mansey
Hello, my application requries opening ssh connections to a list of hosts
(typically less than 10), then performing different commands on each and
parsing the output. So I need a way to programatically address the ssh
commands sent to the correct host. eg execute command x on host y. However
I dont want to open a new connection and log in each time a command is
needed, I want it to log in when first needed and keep the session open for
future use.
it seems fabric's execute command should be able to do this but perhaps im
just missing that there is a far simpler way of making an ssh class that
can be called with a command and an index for host?
so id like to do something like this
env.hosts = [host1, host2, host3]
execute(command1, host1)
execute(commandm, hostn)
Am I on the right track here? Are there any code examples available for this use-case?
Thanks
Jon
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user
Jeff Forcier
2014-03-22 06:16:49 UTC
Permalink
Brandon is correct (thanks Brandon!), Fabric attempts to cache open
connections whenever possible. So you should ideally only be seeing
the bare minimum of network traffic/sessions required. You can inspect
that connections cache Brandon mentioned to see what their current
state is at any time.
Post by Brandon Whaley
Hey Jon,
http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=cache#connections
The live connections are stored in fabric.state.connections for the life of
your program run (I believe, feel free to correct me if I'm wrong there).
Post by Jon Mansey
Hello, my application requries opening ssh connections to a list of hosts
(typically less than 10), then performing different commands on each and
parsing the output. So I need a way to programatically address the ssh
commands sent to the correct host. eg execute command x on host y. However I
dont want to open a new connection and log in each time a command is needed,
I want it to log in when first needed and keep the session open for future
use.
it seems fabric's execute command should be able to do this but perhaps im
just missing that there is a far simpler way of making an ssh class that can
be called with a command and an index for host?
so id like to do something like this
env.hosts = [host1, host2, host3]
execute(command1, host1)
execute(commandm, hostn)
Am I on the right track here? Are there any code examples available for this use-case?
Thanks
Jon
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user
--
Jeff Forcier
Unix sysadmin; Python/Ruby engineer
http://bitprophet.org
Jon Mansey
2014-03-22 20:49:04 UTC
Permalink
Gents, thanks for the feedback, the connections cache is just what I need to avoid setting up and tearing down connections every time I need a command run on a host.

I managed to create a function to do command x on host y and get output, like this:

from fabric.api import env, run, execute, settings, hide

env.hosts = ['host1', 'host2']

def sshcmd():
return run(cmd)

def ssh_execute(sshcmnd, host):
with settings(hide('running', 'commands', 'stderr')):
output = execute(sshcmd, hosts=[host]).values()
return output

cmd = 'uname -n'
print ssh_execute(cmd, 'host2')[0]

However I can’t seem to get it to work without cmd needing to be global. as you can see sshcmnd is not used. I can’t set any args to the sshcmd function call without breaking execute(). if I change this to

def sshcmd(sshcmnd):
return run(cmd)

then try to

execute(sshcmd(sshcmnd), hosts=[host])

it stops with this output

No hosts found. Please specify (single) host string for connection:

I can continue working around this with cmd as global but id prefer not to, if anyone can help out, thanks!

Jon
Post by Jeff Forcier
Brandon is correct (thanks Brandon!), Fabric attempts to cache open
connections whenever possible. So you should ideally only be seeing
the bare minimum of network traffic/sessions required. You can inspect
that connections cache Brandon mentioned to see what their current
state is at any time.
Post by Brandon Whaley
Hey Jon,
http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=cache#connections
The live connections are stored in fabric.state.connections for the life of
your program run (I believe, feel free to correct me if I'm wrong there).
Post by Jon Mansey
Hello, my application requries opening ssh connections to a list of hosts
(typically less than 10), then performing different commands on each and
parsing the output. So I need a way to programatically address the ssh
commands sent to the correct host. eg execute command x on host y. However I
dont want to open a new connection and log in each time a command is needed,
I want it to log in when first needed and keep the session open for future
use.
it seems fabric's execute command should be able to do this but perhaps im
just missing that there is a far simpler way of making an ssh class that can
be called with a command and an index for host?
so id like to do something like this
env.hosts = [host1, host2, host3]
execute(command1, host1)
execute(commandm, hostn)
Am I on the right track here? Are there any code examples available for this use-case?
Thanks
Jon
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user
--
Jeff Forcier
Unix sysadmin; Python/Ruby engineer
http://bitprophet.org
Loading...