Distributing Data, Consistently (3/N)

Last time, we determined that setting up an etcd cluster really wasn't all that difficult. Now we'll actually try to do something with it.

First, lets try getting and setting some values using the etcdctl tool:

[vagrant@turtle1 ~]$ etcdctl set foo bar
bar
[vagrant@turtle1 ~]$ etcdctl get foo
bar
[vagrant@turtle1 ~]$ etcdctl set /services/cdd '[ "turtle1", "turtle2", "turtle3" ]'
[ "turtle1", "turtle2", "turtle3" ]
[vagrant@turtle1 ~]$ etcdctl get /services/cdd
[ "turtle1", "turtle2", "turtle3" ]
[vagrant@turtle1 ~]$ etcdctl ls /
/foo
/services
[vagrant@turtle1 ~]$ etcdctl ls /services/
/services/cdd

So far, so good. We can store/retrieve simple values, store/retrieve small JSON docs. And one of the nice built-ins of etcd is that it understands hierarchical keys automagically. Having determined that etcd is behaving as advertised from the command line, the next step is to try to do something programmatically.

etcd has a fucktonne of drivers for a wide variety of languages. Let's mess around with Python, since all the kids (cool or not) are using it these days. There's an embarassement of riches here, no fewer than 7 different driver packages. Here's a summary:

Driver	Supports V3	Supports Connection Pooling
https://github.com/kragniz/python-etcd3	X
https://github.com/jplana/python-etcd		X
https://github.com/russellhaering/txetcd		X
https://github.com/lisael/aioetcd/
https://github.com/crossbario/txaio-etcd	X
https://github.com/dims/etcd3-gateway	X
https://github.com/gaopeiliang/aioetcd3	X	X

I'd like a drive which supports both connection pooling and etcd V3, though I'm really more concerned with the former since we're trying to build HA systems. It doesn't do you a whole lot of good to have an HA keystore if you're only configured to talk to the node that's down.

aioetcd3 seems like the best fit, but after some experimentation I just couldn't get it to return data. I could see it talking to the cluster, but I wasn't able to get any data back at the programmatic level. This could be due to some subtle incompatibility between the drive and the version of etcd installed, or it could be the result of my complete unfamiliarity with Python's asynchronous IO system.

Next on the list is python-etcd, which I was able to get working without issue. Here's the installation:

yum install -y https://centos7.iuscommunity.org/ius-release.rpm
yum install -y python36u python36u-libs python36u-devel python36u-pip
pip3.6 install python-etcd

and here's some simple messing around via REPL:

[vagrant@turtle1 ~]$ python3.6
Python 3.6.7 (default, Dec  5 2018, 15:02:05)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import etcd
>>> client = client = etcd.Client(host=(('turtle1', 2379), ('turtle2', 2379), ('turtle3', 2379)), allow_reconnect=True)
>>> client.members
{'aeb6950c050a83f3': {'id': 'aeb6950c050a83f3', 'name': 'turtle3', 'peerURLs': ['http://10.0.0.4:2380'], 'clientURLs': ['http://10.0.0.4:2379']}, 'ee274cacce804b21': {'id': 'ee274cacce804b21', 'name': 'turtle1', 'peerURLs': ['http://10.0.0.2:2380'], 'clientURLs': ['http://10.0.0.2:2379']}, 'fe9c64eaf3991d47': {'id': 'fe9c64eaf3991d47', 'name': 'turtle2', 'peerURLs': ['http://10.0.0.3:2380'], 'clientURLs': ['http://10.0.0.3:2379']}}
>>> client.get('/foo').value
'bar'
>>> client.get('/services/cdd').value
'[ "turtle1", "turtle2", "turtle3" ]'
>>> client.read('/services/cdd', wait = True).value
'[ "turtle1", "turtle2", "not a turtle" ]'

That last call blocked (as desired) until I upated the value via etcdctl

So far, so good. We've got a working etcd cluster, and we're successfully able to talk to it via Python. Next up, let's write some real(-ish) code.