[Fab-user] Deploying to multiple sets of hosts

Discussion:

Nathan Nobbe

2013-10-25 22:20:42 UTC

Hi everyone,

I've recently started using Fabric (any Python for that matter)! So far I'm
loving Fabric compared to Capistrano, which is next to useless outside of
Ruby projects that I can tell.

I have hit a snag with Fabric though, and I'm surprised nobody else seems
to have dealt with it. I've got sets of nodes that comprise an
'environment'. For example, 2 boxes are used as db servers, 2 boxes as
application servers (not exactly my setup, but that's the idea).

Now I can have 2 tasks for this with @host decorators that map the hosts to
the deployment logic, however, since there is no way to change the hosts
within a task, I have to make 2 separate calls to fab on the CLI.

The problem is, I have several sets of nodes and the order of operations in
the deployment is itself some logic I would like to encapsulate. Best
approach I have so far is to place the fab calls into a BASH script.

I'm shocked that there's no way to change hosts within a task, and moreover
that this desire hasn't arisen before. Is there some way to change the
hosts mid-task? I can't find anything on Google or Stackoverflow. Or am I
thinking about this the wrong way? Seems like grouping the hosts the way I
have makes sense. I took a look at roles, but I don't think that's any
different than say an @host decorator with the 'db' servers for example.

Your help appreciated!

thanks,

-nathan

Gilgamezh

2013-10-25 23:57:12 UTC

Permalink

Post by Nathan Nobbe
Hi everyone,
I've recently started using Fabric (any Python for that matter)! So far
I'm loving Fabric compared to Capistrano, which is next to useless
outside of Ruby projects that I can tell.
I have hit a snag with Fabric though, and I'm surprised nobody else
seems to have dealt with it. I've got sets of nodes that comprise an
'environment'. For example, 2 boxes are used as db servers, 2 boxes as
application servers (not exactly my setup, but that's the idea).
to the deployment logic, however, since there is no way to change the
hosts within a task, I have to make 2 separate calls to fab on the CLI.
The problem is, I have several sets of nodes and the order of operations
in the deployment is itself some logic I would like to encapsulate. Best
approach I have so far is to place the fab calls into a BASH script.
I'm shocked that there's no way to change hosts within a task, and
moreover that this desire hasn't arisen before. Is there some way to
change the hosts mid-task? I can't find anything on Google or
Stackoverflow. Or am I thinking about this the wrong way? Seems like
grouping the hosts the way I have makes sense. I took a look at roles,
the 'db' servers for example.
Your help appreciated!
thanks,
-nathan

Hi!

roles are in the env dict. You can do this:

env.roledefs.update({
'webserver': ['www1', 'www2'],
'dbserver': ['db1']
})

then.....

for server in env.roledefs['webserver']:
do something
for server in env.roledefs['dbserver']:
do something.

In all the cases you can set a specific host

for example:

with settings(host_string='somehost.domain'):
do some

it's just python! :D

More in:
http://docs.fabfile.org/en/1.8/api/core/decorators.html?highlight=roledefs
http://docs.fabfile.org/en/1.8/usage/env.html?highlight=roledefs
http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=roledefs

regards!

Gilgamezh.

Brandon Whaley

2013-10-26 03:51:09 UTC

Permalink

I prefer using execute:
http://docs.fabfile.org/en/1.8/api/core/tasks.html?highlight=execute#fabric.tasks.execute

Post by Gilgamezh

Hi!
env.roledefs.update({
'webserver': ['www1', 'www2'],
'dbserver': ['db1']
})
then.....
do something
do something.
In all the cases you can set a specific host
do some
it's just python! :D
http://docs.fabfile.org/en/1.**8/api/core/decorators.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/api/core/decorators.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/env.html?highlight=**roledefs<http://docs.fabfile.org/en/1.8/usage/env.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/execution.html?**highlight=roledefs<http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=roledefs>
regards!
Gilgamezh.
______________________________**_________________
Fab-user mailing list
https://lists.nongnu.org/**mailman/listinfo/fab-user<https://lists.nongnu.org/mailman/listinfo/fab-user>

Nathan Nobbe

2013-10-26 06:24:46 UTC

Permalink

Brandon,

Cool stuff, I was able to get this example from the answer on SO

http://stackoverflow.com/questions/15841024/python-fabric-no-host-found-must-manually-set-env-host-string

working pretty easily with multiple hosts, and I see the power here, but
now I'm using python at the command line instead of fab. Can execute be
used within fabric tasks to change the set of hosts and run new actions on
the new hosts?

thanks,

-nathan

Post by Brandon Whaley
http://docs.fabfile.org/en/1.8/api/core/tasks.html?highlight=execute#fabric.tasks.execute

Post by Gilgamezh

Hi!
env.roledefs.update({
'webserver': ['www1', 'www2'],
'dbserver': ['db1']
})
then.....
do something
do something.
In all the cases you can set a specific host
do some
it's just python! :D
http://docs.fabfile.org/en/1.**8/api/core/decorators.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/api/core/decorators.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/env.html?highlight=**roledefs<http://docs.fabfile.org/en/1.8/usage/env.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/execution.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=roledefs>
regards!
Gilgamezh.
______________________________**_________________
Fab-user mailing list
https://lists.nongnu.org/**mailman/listinfo/fab-user<https://lists.nongnu.org/mailman/listinfo/fab-user>

_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user

Brandon Whaley

2013-10-26 15:39:17 UTC

Permalink

Hi Nathan, there are some more detailed use cases of execute here:
http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=execute#intelligently-executing-tasks-with-execute

I use it to (for example) run VPS creation commands on a host node, then
run OS configuration commands on the resulting container once it has been
started. The model you're probably looking for is:

execute(do_work, hosts=host_list)

Where do_work is another fabric task and host_list is just a list of host
strings.

Post by Nathan Nobbe
Brandon,
Cool stuff, I was able to get this example from the answer on SO
http://stackoverflow.com/questions/15841024/python-fabric-no-host-found-must-manually-set-env-host-string
working pretty easily with multiple hosts, and I see the power here, but
now I'm using python at the command line instead of fab. Can execute be
used within fabric tasks to change the set of hosts and run new actions on
the new hosts?
thanks,
-nathan

Post by Brandon Whaley
http://docs.fabfile.org/en/1.8/api/core/tasks.html?highlight=execute#fabric.tasks.execute

Post by Gilgamezh

Hi!
env.roledefs.update({
'webserver': ['www1', 'www2'],
'dbserver': ['db1']
})
then.....
do something
do something.
In all the cases you can set a specific host
do some
it's just python! :D
http://docs.fabfile.org/en/1.**8/api/core/decorators.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/api/core/decorators.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/env.html?highlight=**roledefs<http://docs.fabfile.org/en/1.8/usage/env.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/execution.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=roledefs>
regards!
Gilgamezh.
______________________________**_________________
Fab-user mailing list
https://lists.nongnu.org/**mailman/listinfo/fab-user<https://lists.nongnu.org/mailman/listinfo/fab-user>

_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user

Nathan Nobbe

2013-10-28 18:27:57 UTC

Permalink

Brandon,

This bit

*However, itâs often convenient to wrap up multi-task invocations like this
into their own, âmetaâ tasks.*

describes exactly what I'm looking for; thanks!

Post by Brandon Whaley
http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=execute#intelligently-executing-tasks-with-execute
I use it to (for example) run VPS creation commands on a host node, then
run OS configuration commands on the resulting container once it has been
execute(do_work, hosts=host_list)
Where do_work is another fabric task and host_list is just a list of host
strings.

Post by Brandon Whaley
http://docs.fabfile.org/en/1.8/api/core/tasks.html?highlight=execute#fabric.tasks.execute

Post by Gilgamezh

Hi!
env.roledefs.update({
'webserver': ['www1', 'www2'],
'dbserver': ['db1']
})
then.....
do something
do something.
In all the cases you can set a specific host
do some
it's just python! :D
http://docs.fabfile.org/en/1.**8/api/core/decorators.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/api/core/decorators.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/env.html?highlight=**roledefs<http://docs.fabfile.org/en/1.8/usage/env.html?highlight=roledefs>
http://docs.fabfile.org/en/1.**8/usage/execution.html?**
highlight=roledefs<http://docs.fabfile.org/en/1.8/usage/execution.html?highlight=roledefs>
regards!
Gilgamezh.
______________________________**_________________
Fab-user mailing list
https://lists.nongnu.org/**mailman/listinfo/fab-user<https://lists.nongnu.org/mailman/listinfo/fab-user>

_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user

Nathan Nobbe

2013-10-26 06:01:09 UTC

Permalink

Gilgamezh,

Thanks for your reply. I must be doing something wrong here, when I run
this task it prompts me for a host. This is the problem I've been running
into, where an @hosts decorator seemed to be the only way, but then I can
only bind a single set of hosts.

- testfabenv.py -
from fabric.api import *

mylocalhost = '127.0.0.1'
centoshost = '10.0.2.15'

env.roledefs.update({
'local': [mylocalhost, centoshost],
})

@task
def do_centos_stuff():
for server in env.roledefs['local']:
run('echo hello')

***@gnunix:~/fabric-test$ fab -f testfabenv.py do_centos_stuff
No hosts found. Please specify (single) host string for connection: ^C
Stopped.

-nathan

Post by Nathan Nobbe
Hi everyone,

Post by Nathan Nobbe
I've recently started using Fabric (any Python for that matter)! So far
I'm loving Fabric compared to Capistrano, which is next to useless
outside of Ruby projects that I can tell.
I have hit a snag with Fabric though, and I'm surprised nobody else
seems to have dealt with it. I've got sets of nodes that comprise an
'environment'. For example, 2 boxes are used as db servers, 2 boxes as
application servers (not exactly my setup, but that's the idea).
to the deployment logic, however, since there is no way to change the
hosts within a task, I have to make 2 separate calls to fab on the CLI.
The problem is, I have several sets of nodes and the order of operations
in the deployment is itself some logic I would like to encapsulate. Best
approach I have so far is to place the fab calls into a BASH script.
I'm shocked that there's no way to change hosts within a task, and
moreover that this desire hasn't arisen before. Is there some way to
change the hosts mid-task? I can't find anything on Google or
Stackoverflow. Or am I thinking about this the wrong way? Seems like
grouping the hosts the way I have makes sense. I took a look at roles,
the 'db' servers for example.
Your help appreciated!
thanks,
-nathan