For Hackweek 11 I thought it’d be fun to learn something about creating Android apps. The basic training is pretty straightforward, and the auto-completion (and auto-just-about-everything-else) in Android Studio is excellent. So having created a “hello world” app, and having learned something about activities and application lifecycle, I figured it was time to create something else. Something fun, but something I could reasonably complete in a few days. Given that Android devices are essentially just high res handheld screens with a bit of phone hardware tacked on, it seemed a crime not to write an app that draws something pretty. Continue reading
Since the Senate passed legislation expanding your surveillance powers on Thursday night, you’ve copped an awful lot of flack on Twitter. Part of the problem, I think – aside from the legislation being far too broad – is that we don’t actually know who you are, or what exactly it is you get up to. You could be part of a spy novel, a movie or a decades-long series of cock ups. You could be script kiddies with a budget. Or you could be something else entirely.
At times like this I try to remind myself to assume good faith; to remember that most people are basically decent and are trying to live a good life. Some people are even trying to make the world a better place, whatever that might mean.
For those of you then who are decent people, and who are trying to keep Australia safe from whatever mysterious threats are out there that we don’t know about – all without wishing to impinge on or risk destroying the freedoms that we enjoy here – you have my thanks.
For those of you involved in the formulation of The National Security Legislation Amendment Bill 2014 (No 1) – you who might be reading this post as I type it, rather than after I publish it – I have tried very, very hard to imagine that you honestly believe you are making the world a better place. And maybe you do actually think that, but for my part I cannot see the powers granted as anything other than a direct assault on our democracy. As Glenn Greenwald pointed out, I should be more worried about bathroom accidents, restaurant meals and lightning strikes than terrorism. As a careful bath user with a strong stomach and a sturdy house to hide in, I think I’m fairly safe on that front. Frankly I’m more worried about climate change. Do you have anyone on staff who can investigate that threat to our national security?
Anyway, thanks for reading, and I’ll take it as a kindness if you don’t edit this post without asking first.
The Australian government often engages in public consultation on a variety of matters. This is a good thing, because it provides an opportunity for us to participate in our governance. One such recent consultation was from the Attorney-General’s Department on Online Copyright Infringement. I quote:
On 30 July 2014, the Attorney-General, Senator the Hon George Brandis, and the Minister for Communications Malcolm Turnbull MP released a discussion paper on online copyright infringement.
Submissions were sought from interested organisations and individuals on the questions outlined in the discussion paper and on other possible approaches to address this issue.
Submissions were accepted via email, and there was even a handy online form where you could just punch in your answers to the questions provided. The original statement on publishing submissions read:
Submissions received may be made public on this website unless otherwise specified. Submitters should indicate whether any part of the content should not be disclosed to the public. Where confidentiality is requested, submitters are encouraged to provide a public version that can be made available.
This has since been changed to:
Submissions received from peak industry groups, companies, academics and non-government organisations that have not requested confidentiality are being progressively published on the Online copyright infringement—submissions page.
As someone who in a fit of inspiration late one night (well, a fit of some sort, but I’ll call it inspiration), put in an individual submission I am deeply disappointed that submissions from individuals are apparently not being published. Geordie Guy has since put in a Freedom of Information request for all individual submissions, but honestly the AGD should be publishing these. It was after all a public consultation.
For the record then, here’s my submission:
- An update from the OpenStack Foundation (presented by me, with slides provided by the Foundation).
- A talk about the NeCTAR cloud and using the command line tools to work with images, by Scott Bragg.
- A talk on spinning up instances with Nova and Heat, by Stewart Wilde.
- A talk by me on Ceph, and how it can be used as the storage backend for an OpenStack cloud.
We also had some posters, stickers and fridge magnets made up. The fridge magnets were remarkably popular. If you weren’t at TasLUG last night, and you want a fridge magnet, first download this image (the full-res one linked to, not the inline one):
Then, go to Vistaprint and place an order for Magnetic Business Cards, using this image. You can get 25 done for about $10, plus shipping.
Finally, I would like to publicly thank the OpenStack Foundation for supporting this event.
To Whom It May Concern,
Please allow me to offer my congratulations on your election to office. Depending on the exact circumstances, this is a testament either to your general excellence, or to the deep incompetence of your opponents (or possibly to a combination of the two).
I would like to offer – if I may – a single piece of advice: do not, under any circumstances, publicly complain about the inadequacies of your predecessors or the mess they have left. You will sound like a petulantly whining child. Rather, explain plainly the state of things as they are, and your bold plan to bring us to a better future. Many of us are not idiots, and will respect (and even vote for) competent adults, assuming we are able to find any.
All the best for the next few years.
Some time late last Sunday night, I stumbled upon a
fight discussion on Twitter. It turns out there are actually more threads to it than I’ve reproduced here, so if you’re really keen, do feel free to click through on individual tweets to see where it went. Here I’ve only reproduced the thread I read at the time. It starts with this:
My wife and I have been making our own bacon since January this year. The year is almost over so it’s well past time I documented the process for posterity, especially since I expect my typing to remain legible for longer than my handwriting. Also, I know there’s at least a couple of people out there who are actually interested in this
In 1996 Regurgitator released a song called “Kong Foo Sing“. It starts with the line “Happiness is a Kong Foo Sing”, in reference to a particular brand of fortune cookie. But one night last week at the OpenStack Summit, I couldn’t help but think it would be better stated as “Happiness is a Hong Kong SIM”, because I’ve apparently become thoroughly addicted to my data connection.
I was there with five other SUSE engineers who work on SUSE Cloud (our OpenStack offering); Ralf Haferkamp, Michal Jura, Dirk Müller, Vincent Untz and Bernhard Wiedemann. We also had SUSE crew manning a booth which had one of those skill tester machines filled with plush Geekos. I didn’t manage to get one. Apparently my manual dexterity is less well developed than my hacking skills, because I did make ATC thanks to a handful of openSUSE-related commits to TripleO (apologies for the shameless self-aggrandizement, but this is my blog after all).
Given this was my first design summit, I thought it most sensible to first attend “Design Summit 101“, to get a handle on the format. The summit as a whole is split into general sessions and design summit sessions, the former for everyone, the latter intended for developers to map out what needs to happen for the next release. There’s also vendor booths in the main hall.
Roughly speaking, design sessions get a bunch of people together with a moderator/leader and an etherpad up on a projector, which anyone can edit. Then whatever the topic is, is hashed out over the next forty-odd minutes. It’s actually a really good format. The sessions I was in, anyone who wanted to speak or had something to offer, was heard. Everyone was courteous, and very welcoming of input, and of newcomers. Actually, as I remarked on the last day towards the end of Joshua McKenty’s “Culture, Code, Community and Conway” talk, everyone is terrifyingly happy. And this is not normal, but it’s a good thing.
As I’ve been doing high availability and storage for the past several years, and have also spent time on SUSE porting and scalability work on Crowbar, I split my time largely between HA, storage and deployment sessions.
On the deployment front, I went to:
- HA/Production Configuration, where the pieces of OpenStack that TripleO needs to deploy in a highly available manner were discussed (actually this could have been discussed for a solid week
- Stable Branch Support and Update Futures, about updating images made for TripleO.
- An Evaluation of OpenStack Deployment Frameworks, where two guys from Symantec discussed the evaluation they’d done of Fuel, JuJu/MaaS, Crowbar, Foreman and Rackspace Private Cloud. In short, nothing was perfect, but Crowbar 1.6 performed the best (i.e. met their requirements better than any of the other solutions tested).
- Roundtable: Deploying and Upgrading OpenStack.
- OpenStack’s Bare Metal Provisioning Service, wherein I attained a better understanding of Ironic.
- It Not Just An Unicorn, Updating Our Public Cloud Platform from Folsom to Grizzly - how eNovance manage upgrades. Automate all the things and test, test, test. Binary updates are done by rsyncing prepared trees, but everything can be rolled back and forwards, because everything is in revision control. It sounds like they’ve done a very thorough job in their environment. I’m less sure this technique is applicable in a generic fashion.
- The Road to Live Upgrades. Notably they want to add a live upgrade test as a commit gate.
- Hardware Management Ramdisk. Lots of work to do here for Ironic to deploy ramdisks to do, e.g.: firmware updates, RAID configuration, etc.
- Firmware Updates (followed right on from the previous session).
- Making Ironic Resilient to Failures (what do you do if your TFTP/PXE server goes away?)
- Compass – Yet Another OpenStack Deployment System, from Huawei, to be released as open source under the Apache 2.0 license “soon” (end of November). A layer on top of Chef, but with other configuration tools as pluggable modules. If you squint at it just right, I’d argue it’s not dissimilar to Crowbar, at least from a high level.
On High Availability:
- Practical Lessons from Building a Highly Available OpenStack Private Cloud (Ceph for all storage, HA via four separate Pacemaker clusters. Notably the cluster running compute can scale out by just adding more nodes.
- High Availability Update: Havana and Icehouse, wherein I attempted to look scary sitting the front row wearing my STONITH Deathmatch t-shirt. I hope Florian and Syed will forgive my butchering their talk by summarizing it as: If you’re using MySQL, you want Galera. RabbitMQ still has consistency issues with mirrored queues and there can be only one Neutron L3 agent, so you need Pacemaker for those at least, so using Pacemaker to “HA all the things” is still an eminently reasonable approach (haproxy is great for load balancing, but no good if you have a service that’s fundamentally active/passive). Use Ceph for all your storage.
- Database Clusters as a Service in OpenStack: Integrated, Scalable, Highly Available and Secure. Focused on MySQL/MariaDB/Percona, Galera and variants thereof, which combinations are supported by Rackspace, HP Cloud and Amazon, and various deployment considerations (including replication across data centers).
- Encrypted Block Storage: Technical Walkthrough. This looks pretty neat. Crypto is done on the compute host via dm-crypt, so everything is encrypted in the volume store and even over the wire going to and from the compute host. Still needs work (naturally), notably it currently uses a single static key. Later, it will use Barbican.
- Swift Drive Workloads and Kinetic Open Storage. Sadly I had to skip out of this one early, but Seagate now have an interesting product which is a disk (and some enclosures) which present disks as key/value stores over ethernet, rather than as block devices. The idea here is you remove a whole lot of layers of the storage stack to try to get better performance.
- Real World Usage of GlusterFS + OpenStack. Interesting history of the project, what the pieces are, and how they now provide an “all-in-one” storage solution for OpenStack.
- Ceph: The De Facto Storage Backend for OpenStack. It was inevitable that this would go back-to-back with a GlusterFS presentation. All storage components (Glance, Cinder, object store) unified. Interestingly the
libvirt_image_type=rbdoption lets you directly boot all VMs from Ceph (at least if you’re using KVM). Is it the perfect stack? “Almost” (glance images are still copied around more than they should be, but there’s a patch for this floating around somewhere, also some snapshot integration work is still necessary).
- Sheepdog: Yet Another All-In-One Storage for Openstack. So everyone is doing all-in-one storage for OpenStack now I haven’t spent any time with Sheepdog in the past, so this was interesting. It apparently tries to have minimal assumptions about the underlying kernel and filesystem, yet supports thousands of nodes, is purportedly fast and small (<50MB memory footprint) and consists of only 35K lines of C code.
- Ceph OpenStack Integration Unconference (gathering ideas to improve Ceph integration in OpenStack).
Around all this of course were many interesting discussions, meals and drinks with all sorts of people; my immediate colleagues, my some-time partners in crime, various long-time conference buddies and an assortment of delightful (and occasionally crazy) new acquaintances. If you’ve made it this far and haven’t been to an OpenStack summit yet, try to get to Atlanta in six months or Paris in a year. I don’t know yet whether or not I’ll be there, but I can pretty much guarantee you’ll still have a good time.
Having published a transcript of the talk I gave last year at OSDC 2012, I stupidly decided I’d better do the same thing with this year’s OSDC 2013 talk. It’s been edited slightly for clarity (for example, you’ll find very few “ums” in the transcript) but should otherwise be a reasonably faithful reproduction.
It’s Hack Week again. This time around I decided to look at running TripleO on openSUSE. If you’re not familiar with TripleO, it’s short for OpenStack on OpenStack, i.e. it’s a project to deploy OpenStack clouds on bare metal, using the components of OpenStack itself to do the work. I take some delight in bootstrapping of this nature – I think there’s a nice symmetry to it. Or, possibly, I’m just perverse.
Anyway, onwards. I had a chat to Robert Collins about TripleO while at PyCon AU 2013. He introduced me to diskimage-builder and suggested that making it capable of building openSUSE images would be a good first step. It turned out that making diskimage-builder actually run on openSUSE was probably a better first step, but I managed to get most of that out of the way in a random fit of hackery a couple of months ago. Further testing this week uncovered a few more minor kinks, two of which I’ve fixed here and here. It’s always the cross-distro work that seems to bring out the edge cases.
Then I figured there’s not much point making diskimage-builder create openSUSE images without knowing I can set up some sort of environment to validate them. So I’ve spent large parts of the last couple of days working my way through the TripleO Dev/Test instructions, deploying the default Ubuntu images with my openSUSE 12.3 desktop as VM host. For those following along at home the
install-dependencies script doesn’t work on openSUSE (some manual intervention required, which I’ll try to either fix, document, or both, later). Anyway, at some point last night, I had what appeared to be a working seed VM, and a broken undercloud VM which was choking during
Calling http://169.254.169.254/2009-04-04/meta-data/instance-id' failed Request timed out
Figuring that out, well… There I was with a seed VM deployed from an image built with some scripts from several git repositories, automatically configured to run even more pieces of OpenStack than I’ve spoken about before, which in turn had attempted to deploy a second VM, which wanted to connect back to the first over a virtual bridge and via the magic of some iptables rules and I was running tcpdump and tailing logs and all the moving parts were just suddenly this GIANT COSMIC DANCE in a tiny little box on my desk on a hill on an island at the bottom of the world.
It was at this point I realised I had probably been sitting at my computer for too long.
It turns out the problem above was due to
my_ip being set to an empty string in
/etc/nova/nova.conf on the seed VM. Somehow I didn’t have the fix in my local source repo. An additional problem is that libvirt on openSUSE, like Fedora, doesn’t set
uri_default="qemu:///system". This causes nova baremetal calls from the seed VM to the host to fail as mentioned in bug #1226310. This bug is apparently fixed, but apparently the fix doesn’t work for me (another thing to investigate), so I went with the workaround of putting
So now (after a rather spectacular amount of disk and CPU thrashing) there are three OpenStack clouds running on my desktop PC. No smoke has come out.
- The seed VM has successfully spun up the “baremetal_0″ undercloud VM and deployed OpenStack to it.
- The undercloud VM has successfully spun up the “baremetal_1″ and “baremetal_2″ VMs and deployed them as the overcloud control and compute nodes.
- I have apparently booted a demo VM in the overcloud, i.e. I’ve got a VM running inside a VM, although I haven’t quite managed to ssh into the latter yet (I suspect I’m missing a route or a firewall rule somewhere).
I think I had it right last night. There is a giant cosmic dance being performed in a tiny little box on my desk on a hill on an island at the bottom of the world.
Or, I’ve been sitting at my computer for too long again.