[Fab-user] fabric, invoke - scaling up to support 100 hosts & SSH keys ?

[Doug Vos]

OK, upgraded everything in my DEV environment to the latest versions.

*What's a good strategy for scaling up to support 100 hosts & SSH keys?*

My production environment uses a ~/.ssh/config file (with about 20 entries)

*Can't get ~/.ssh/config working with Fabric 2.3.1 yet. (new DEV) **It's
not working yet, sigh...*
*Should I expect **~/.ssh/config **to work with Fabric 2.X
<http://docs.fabfile.org/en/latest/concepts/configuration.html#loading-and-using-ssh-config-files>?*
What about YAML file to bootstrap testing? YAML or JSON to get my DEV going?
Could something about my activated MiniConda DEV environment -- or the way
Fabric's default config looks at the environment be causing me trouble?


*Please advise.*

*Current production is running well (using **~/.ssh/config) *-- but has a
few nasty passphrase and SSH issues...

- Python 2.7.14
- Fabric 1.14.0
- Paramiko 2.4.0


*New DEV is running:*

- Python 3.6.6 :: Anaconda, Inc. ($ activate py36fabric )
- Fabric 2.3.1
- Paramiko 2.4.1
- Invoke 1.1.1

[Jeff Forcier]

Hi Doug,

If you've got a working Fabric 1 setup, feel free to continue using that
for another few months or so and just check in on our changelog
periodically to see what features have been newly ported over. As-is, the
2.x line is still more of a technology/API preview than a full "port all
the way over" 100% rework.

That said, I do have some questions/answers!

- Paramiko-based kit in general should absolutely scale up to 100+
hosts/keys, e.g. Fabric 1 has been proven to work with thousands of hosts
or more.
- Fabric 2 is still fleshing out 'blessed'/default modes of configuring
your target hosts and their parameters, as well as kicking the tires of the
default thread-based concurrency on offer - but if you create your own
Connection objects from some arbitrary data structures of your own, they
*ought* to already scale up to a large number of hosts.
- Can you post a sanitized copy of your SSH config file & details on what
exactly is "not working"? _Most_ of what v1 supported in SSH config files
(which was a relatively small % of total ssh_config capability, granted)
should already be supported in v2, as long as you're not explicitly
disabling SSH config loading.
- In terms of config files, we support YAML, JSON or Python modules and
they're all considered equal in terms of support (aside from some minor
per-format issues, such as Python import shenanigans for .py files).
- It's 100% likely that anything we settle on for a default
store-your-host-data scheme will be phrased generically and will support
all of those formats too.
- We're also going to add more formats too, eg there's an open ticket for
adding TOML.
- The dev environment/IDE shouldn't matter too much; the primary shell env
vars that Fabric/Invoke look at are ones prefixed with INVOKE_xxx or
FABRIC_xxx.

Best,
Jeff

Post by Doug Vos
OK, upgraded everything in my DEV environment to the latest versions.
*What's a good strategy for scaling up to support 100 hosts & SSH keys?*
My production environment uses a ~/.ssh/config file (with about 20 entries)
*Can't get ~/.ssh/config working with Fabric 2.3.1 yet. (new DEV) **It's
not working yet, sigh...*
*Should I expect **~/.ssh/config **to work with Fabric 2.X
<http://docs.fabfile.org/en/latest/concepts/configuration.html#loading-and-using-ssh-config-files>?*
What about YAML file to bootstrap testing? YAML or JSON to get my DEV going?
Could something about my activated MiniConda DEV environment -- or the way
Fabric's default config looks at the environment be causing me trouble?
*Please advise.*
*Current production is running well (using **~/.ssh/config) *-- but has a
few nasty passphrase and SSH issues...
- Python 2.7.14
- Fabric 1.14.0
- Paramiko 2.4.0
*New DEV is running:*
- Python 3.6.6 :: Anaconda, Inc. ($ activate py36fabric )
- Fabric 2.3.1
- Paramiko 2.4.1
- Invoke 1.1.1
_______________________________________________
Fab-user mailing list
https://lists.nongnu.org/mailman/listinfo/fab-user