Russel Winder's Website


Learning Programming

On twitter recently, someone asked me:

What was your XP picking up languages and paradigms?

How did you discover the value of new concepts you decided to learn?

I wrote an email in reply. It got a bit long for an email, and it began to look like a blog post. So I copy and pasted the text and worked on it till this blog post resulted.


I think the most important thing for me in learning programming was that I was a theoretical physicist and scientist bringing a scientific method approach to the whole activity. I first learned FORTRAN programming using punch cards sent off to a computer with a one week turnaround. This forces you to carefully mentally execute your code. This is I think a skill fewer and fewer people practice.

I then moved on to having direct access to job submission, but still using FORTRAN on cards, but now with a one day turnaround. In effect no change: in terms of development a week and a day leave you with the lack of continuity of thinking. I did though learn about abstract data types as a concept around then. I mock up this idea in FORTRAN for my code. OK in hindsight I was hacking the language to achieve a goal the language was not designed for, and yet my code was easier to evolve with those constructs for the uses I was making of them. Other people programming around me were still working with the standard FORTRAN model and doing much more debugging and getting wrong results they didn’t know were wrong.

Around this time I did some work that involved:

  1. Accessing mainframe codes using editor and job control submitted for day turnaround batch via paper tape. Yuk. I grumbled so much and pointed at a minicomputer in the room so that;

  2. Using a rewrite of the FORTRAN code in Basic on aforementioned local mini-computer. Still paper tape but the turnaround time was fabulous. I had to learn the magic skills of very thin sticky tape to block up holes without blocking the fast tape reader and a hole punch to put corrections in.

So then we are on to the terminal access to medium and big local and remote computers. The crucial thing here is quasi-instant turnaround. Actually still batch job submission, but it was all on the machines so you could track the job and thus schedule your time. Furthermore you could have one job waiting and work on preparing another. It was around this time I got obsessed with program development process and tooling. And learning the subtleties of the only error message IBM machines ever give: 0c4.

Next stage was learning new languages. First stop Algol 68. This was an eye opener, and a revelation. Everything I had been tinkering with using FORTRAN suddenly became easy. Pascal was clearly a failure in comparison to Algol 68 but it had to be used because it had to be used. Except FORTRAN was still the tool. Along with Schoonership and Reduce. Also a couple of assembly language courses, to find out how computer hardware actually worked.

However I then switched career and taught myself C. It was all about hardware, mostly because I was doing systems programming, but C was so much better than assembly language for this. And UNIX 6 on PDP-11. Joyous. With the move to VAX-11 750 things got interesting: octal → hexadecimal. But it was all about making hardware work, but using nice, elegant, understandable C code. I also got involved in some UI and quasi-IDE research work. This was right at the start of the graphical display revolution so everything was actually quite simple but revolutionary. And very exciting. Also learned Prolog which was an eye opener: script submitted to compute engine, a whole new way of thinking.

Also worth noting that editing and executing code instant turnaround, no batch any more. Bugs instantly available. I stopped being careful about carefully mentally executing my code. This was an error. I try harder since then, but the instant turnaround (which is today even more instant) made development time longer because of all the trivial errors.

Then the academic phase teaching in historical order: FORTRAN, Pascal, C++, Scheme, Miranda (which with KRC led to Haskell), Fortran, Prolog, Ada, and Java. The theme here is having to learn new paradigms of computation and teach them professionally to others. Also I was researching psychology of programming, UI, and parallel programming and programming languages. I was part of teams inventing languages. Mostly OO as I was in the OO not functional camp during the great OO vs FP war of the 1980s.

Along with the plethora of languages and computational models, and always present, was my obsession with build automation, and scientific method as an approach to debugging. 1979 was a revolutionary year for me as I found out about and was able to use Make. Till then I have been tinkering creating my own scripts for automating build. Make changed my life.

For the last 20 years it has just been "more of the same". The languages change (and mostly improve), IDE and editors change (and mostly improve), but my mental structures for dealing with it all have not really changed. Yes the facts and beliefs evolve, but the meta-facts and meta-beliefs have stayed fairly constant. Yes, I know that people cannot reason about their own thinking, but I have not had any serious events of crisis of belief about programming and software development in the last 20 years.

The dimensions are simplicity, or at least understandability, symmetry, or at least understandable lack of it, and performance, or understanding lack of it. I am in the "OO failed and FP failed but the amalgam of OO and FP has moved us forward massively" camp. For this we have to thank Scala for providing the bridge in the early 2000s. Also C++ moving from OO to generic as it’s label, and that Go and Rust have eschewed classes but are still object-based languages.

Having a knowledge of processor architectures and executions up to high level programming, with a knowledge of many different paradigms of software execution is for me the core, along with a demand for scientific method and not guesswork.

MaRIONet First Meeting

Entirely by accident (a retweet of something that I just happened to see) I found out about the first meeting of a new EPSRC network, MaRIONet (Manycore Research, Innovation, and Opportunities Network). The meeting was yesterday (a day I had free) at Imperial College, a place I can get to fairly easily – even during a tube a train strike. So I signed up and went along.

The opening keynote for the day was by Steve Furber and was rather good. He started with a history (by decades) of computing in Manchester, UK. This turns out to be an interesting list of firsts:

  • Baby 1948, the first stored program machine.

  • 1950s, the Mark 1, the first real computer.

  • 1960s, Atlas, the first virtual memory computer.

  • 1970s, MU5, not so much a first per se, but the basis of the workhorse computer for national and local government for a long while.

  • 1980s, the Dataflow machine, the first dataflow hardware machine – in the end not a success but led to all the software dataflow architectures so common today.

  • 1990s, Amulet, the beginning of the ARM core era.

  • 2000s, SpiNNaker, the beginning of real-time brain modelling, the beginning of small core, tightly bound CPU and memory, emphasis on routing small packets.

  • 2010s SpiNNaker 2, more of the same but totally different internals.

A couple of asides made the history fascinating:

  1. It was the Baby machine that got Alan Turing to be in Manchester, he didn’t write any programs for the machine but he did some of the documentation. He actually wrote some papers predicting the future of computers including that computers would have Gigabytes of memory within 60 years, and interesting prediction in 1948 with Baby having a few hundred bytes.

  2. IBM bought the patents on virtual memory from Manchester University for far too little money.

This viewpoint on the history of computing contrasts massively to the usual USA-centric history which invariably ignores any and all contributions not from within the USA.

The other main "take away" for me from this presentation was the need to move away from the multi-core-CPU–bus–Memory architecture we currently have, towards a CPU/Memory/routing network all on chip with no cache. The SpiNNaker machines rely on this sort of many-core approach to have a multi-level (NUMA?) architecture. Others have trodden this road, for example Adepteva, and indeed Intel with their 80-core chip. Sadly I suspect the world will be stuck with x86_64 architectures for workstations and laptops for many years to come, which will inhibit the progress of software for a very long while.

After a short drinks break, there was a period of lightning talks, attenders had the opportunity to "pitch" themselves and their work. Whilst the timetable scheduled academic and industrial, this was a fundamentally academic audience, with just a couple of "industrials", only one of which was really into the purpose of the meeting and the network as a whole. This was followed by a meet and greet, networking session, and then lunch and a bit more networking.

Adam Luqmani from EPSRC then gave a presentation on EPSRCs priorities for the next period of funding rounds and the relationship of network (such as MaRIONet) and what EPSRC likes to see from them. This was clearly pitched at the academic audience, who were clearly seeing indications on what has to be done in the future to obtain research grants.

The rest of the day got rather ruined by a fire alarm. However we then went on to a rather splendid supper and much informal conversation. Overall it was worth going to the meeting. Hopefully I get to hear about future ones.

On Turning Points in Life

For some bizarre reason I am obsessing recently that it snowed. Not much interesting in that per se, this is UK, it snows here. Sometimes anyway. However, I am thinking Brighton 1975-06-02 (turns out it was a Monday but I had to look that up). Yes, the south coast of England in the period leading up to a summer solstice. It snowed. Quite heavily in fact, though it didn’t really settle. I was walking across University of Sussex campus, not really surprising since I was a student there at the time, and it was snowing. Snowing. Cricket season. Snowing.

You are, of course, thinking: where are the photos to prove the claim. Remember though this is 1975. No smartphones. No digital cameras. No money. It was another five years till I got a Canon AE-1 (which got stolen a couple years later, enabling me to trade up to a Canon A-1). So the only proof I can offer is this Met Office report (PDF download) that I found.

I was walking the path from the student union building, in front of the Library, heading back to Norwich House (in which I had a room) thinking "it’s @@@@@@@ snowing, what the @@@@". But life went on, even though it was not possible to throw snowballs at each other.

Whenever I think of this event I am immediately drawn back a few months earlier to a Sunday afternoon (I have no idea of the date, but I am fairly sure it was November 1974 for no reason I can think of) and The Annie Nightingale show. It’s Sunday afternoon, in late autumn, but it wasn’t snowing. No cricket game, no nets. No kung-fu session. So clearly time to do the week’s "homework". No record player. No reel-to-reel tape player. No compact cassette player. Just a guitar. In case you haven’t tried it (I have on many occasions), it is seemingly impossible to play guitar and write papers on quantum mechanics at the same time. So no ability to play proper music. Just a radio. But then Annie Nightingale plays Hawkwind’s "You Shouldn’t Do That" from "In Search of Space". All of it. This is a 15:42 piece, played uninterrupted, in it’s entirety, on national radio – national FM radio, no DAB or DAB+ back then. This piece was then, and remains now, one of my favourites. I entered into a "peak state", have no idea what happened the rest of the afternoon, other than having two completed papers by dinner time. I got As (brag, brag).

This moment connected being at home with family and all my music available, to being on my own away from home, without my music collection, and yet getting stuff done successfully. It was a bridging moment between past and future. A future that is now the past obviously. I have though never, ever not had my music collection ever since that year.

She will almost certainly never read this piece; nor likely ever even know of its existence, but I want to publicly thank Annie Nightingale for keeping me on track with life via her Sunday Afternoon Show.


PS How do you distinguish memories of things that happened from remembered dreams?


PPS This post has no cosmic significance, it is pure personal memory of a time when life was fun, and the only stress was passing exams. Well that and bowling well enough in games, so that we won the competitions.

µCon 2016

Monday and Tuesday this week was µCon 2016. I found much good, a little not so good, and two really outstanding sessions. One obviously was mine :-) The other was the one by Adrian Colyer. You can see the video on Skills Matter’s website here.

I have uploaded the slides of my session to SpeakerDeck version and SlideShare version. However, as always with my presentations these days, there is a lot of doodling on the slides as I present. This means the slide PDFs are only really part of the story. I never watch videos of myself so I have no idea how good or bad the session was but Skills Matter video is available here.

PyCon UK Sprint: Supercomputer in a Briefcase

Yesterday was the sprint day for PyCon UK 2016. I proposed to sprint on "Supercomputer in a Briefcase". I got four helpers, Barry Scott, Emma Gordon, Tom Viner and John Chandler. Richard Wenner offered us use of 10 RaspberryPis (RPis) and a very big hub, but in the end we didn’t use these, we just used the two RPis, hub and DHCP server I had brought. If we do something with this project next year I do hope Richard is around with his RPis.

First technical problem to tackle was which repository to work with. I had assumed Git and had made repositories on GitHub, GitLab, and Bitbucket in preparation. One person voted for Mercurial on BitBucket, but in the end the path of least resistance was GitHub – even though I wanted to go with GitLab. You can get to the repository by following this link.

The first real problem to address was finding out what computers were available on the network – it is assumed that the cluster will always be on a private network. Whilst there are many RPi cluster projects around the world (do a Google search and you can’t miss them) they all put RPis into a rack of some sort, i.e. they are in a fixed configuration. A goal of "Supercomputer in a Briefcase" is to have a dynamic cluster: the cluster has to be able to deal with nodes being taken away and being added. I had previously created a discovery program using scapy and reverse ARP calls. This however required super-user permission and was somewhat unsatisfactory. The team came up with the idea of using Avahi instead. After various experiments, we found that by placing an XML service specification in the right place and restarting the Avahi server we got service notification. This deals with adding nodes to the cluster and of nodes removing themselves, but leaves the problem of nodes being disconnected by having the Ethernet removed or by being powered off. A watchdog will have to be created to cover this situation.

For now the idea is to use OpenMPI as a code and data transfer layer: python3-mpi4py is available on Rasbian and it is based on OpenMPI. The single master node of the cluster starts a job and collects together the result. For us at the sprint the master node was a laptop that was connected to both the cluster network and the rest of the Internet and Web. The master node has to know the collection of computers in the cluster and so the service discovery code was extended to write the file needed by OpenMPI. This covers the case of a fixed network, it will have to be extended to deal with a dynamic network.

People had to leave a little early so the technical effort was stopped there and notes put in place to outline what had been achieved and what needed to be achieved. This has created an excellent platform to progress this idea. Huge thanks to Tom, Emma, John, and Barry for taking the time to contribute. I will continue the work, if others want to join in please head over to Supercomputer in a Briefcase.

PyCon UK 2016 Session

Earlier today I did a very rapid presentation in a somewhat confrontational style. The premise was that all the efforts to make Python code fast of execution was a waste of time and effort: Cython and Numba are mis-directed in trying to treat a dynamic programming language as a static native code language. NumPy provides a C supposedly opaque data structure to be treated as a block box used by employing higher order functions. Yet it too is relatively slow and constraining. The success of Pandas and SciPy, amongst others, in terms of traction and usage hides that, whilst this is a perfect solution for some use cases and users, it is stopping many from achieving serious computations in a short time. Due to the presentation being a short one, the story line leading to Chapel had to be very truncated. I ran some sequential Python code and then some sequential Chapel code which ran about 100 times faster. I then showed a parallel Chapel code which ran 8 times faster because there were 8 cores. The parallel Chapel code was so simple I left it displayed and stopped the presentation. The codes were drawn from the Git repository on GitHub. The slides I uploaded to SpeakerDeck and SlideShare. They are also here.

Update: PyConUK media people have uploaded a video of this session here. I haven’t watched it so have no idea if I like the session or not. Lot’s of people have told me they liked it, so it can’t be too bad.

Ad Hoc Clusters of RaspberryPis

Introduction

At PyConUK 2016 it is planned to run a sprint on Monday 2016-09-19 to create and use a "Supercomputer in a Briefcase". There will be an opportunity to do some pre-planning as there is a short session scheduled for Thursday 2016-09-15T10:30+01:00.

Rationale

Trying to create a supercomputer using RaspberryPis is clearly impossible. Yet people have tried it before (sort of). For example:

All of these are projects clustering RaspberryPis together in a network to create a parallel processing computer – not a supercomputer per se, but a small scale parallel computer. The clusters in these cases have been created with fixed infrastructure. This makes certain aspects of creating a cluster easier, however it requires more hardware, and it ties up the RaspberryPis.

Many places (schools, etc.) have a number of RaspberryPis normally set out with monitors, keyboards, and mice as individual workstations. Usually these workstations are connected to the Internet, or at least in an internal network. They are effectively already in a cluster. This then raises the question: is it possible to use this collection of workstations as a single parallel computer. Clearly the answer must be yes – there is no effective difference between a collection of networked workstations and a dedicated cluster. Being used in this cluster way, the monitors, keyboards, and mice are redundant. So can we then store the RaspberryPis and networking bits and pieces into a briefcase so we have "a supercomputer in a briefcase"? Definitely.

What is going to be done at the sprint?

On the assumption that we have some RaspberryPis and laptops and a private network with all the devices connected, we have to write some software to discover what the nodes on the network are. We will also need some software to push bits of the overall job out to the nodes, and collect the results. We are also going to need some programs to try out.

We might want to try executing the code on a single RaspberryPi and then the cluster to see how much faster (or slower!) the parallel version runs compared to the sequential (run on a single processor) version.

The reason…

The thinking behind all this, is that it would be good for schools to have the ability to introduce students to some of the more straightforward ideas of parallelism in computing and programming. This is not to have these people try and do "supercomputing", or "Big Data", per se, but to introduce them to some of the core ideas in those topics. Data science, business intelligence, etc. all depend on processing large amounts of data and to be done in reasonable time parallelism is required of the computer. With the post-2014 computer science curriculum, topics that were once seen as having to be introduced as final year of university study, can have initial introductions at probably sixth form level to be taken much further in university courses.

Endnote

In principle, we can have anything as nodes in the network as long as they are in the network and can run programs. So we can use laptops, high-end smartphones, etc. as well as RaspberryPis.

And Finally…

Come and join in.

Testing Propositions

During a conversation a couple of months ago, I was waxing lyrical about the rise of proposition-based testing. QuickCheck has been around and used in the Haskell community for quite a long while. The ideas are now though rapidly appearing in other programming languages. It used to be that every programming language had to have an example-based test framework, often based on the SUnit architectecture carried over to JUnit and then a variant in most other languages. It is now the case that every programming language is trying to have a port of QuickCheck, or at least something very similar. Certainly there is Hypothesis for Python (and possibly soon for Java), ScalaCheck (often used from ScalaTest) for Scala, KotlinTest supports property-based testing in Kotlin, Specks for Ceylon. In fact the list is getting longer and longer as QuickCheck-inspired, proposition-based testing gains traction in the mainstream.

It was suggested I should write a brief introductory article about this stuff. So I did. Except that it got a bit longer than originally anticipated. In this month's (August) Overload there is the article "Testing Propositions"[Overload 134, pp17–27, 2016-08]. Overload is a paper journal so you should go and buy it, or become a member of ACCU and get it that way. ACCU chooses to publish the journal electronically for free, so you can get a copy of the PDF from here or see the HTML rendering here. But you should join ACCU

If you have any comments about my article, email me.

Supercomputer in a Case

I will be leading a sprint session at PyConUK 2016 entitled "Supercomputer in a case". This blog is the first of a series trying to explain the idea, and thus introduce the sprint.

I suspect the first question is: what is a supercomputer? Rapidly followed by the second question: what is a case?

What is a Supercomputer?

If we ask Wikipedia, we get the answer:

A supercomputer is a computer with a high-level computational capacity compared to a general-purpose computer.

which, sadly, begs so many questions. However reading through that article there is a lot of good material surrounding what a 'supercomputer' is compared to a 'computer', and introducing a few examples from over the years.

Trying to summarize in a different way: supercomputers are computers that compute hugely bigger problems vastly quicker than your laptop or desktop ever could. Supercomputers are generally massive, require huge amounts of electricity, and thus cost more than anyone except governments and huge companies can afford. In the early days (1960s, e.g. Cray-1) supercomputers were specially designed to be significantly faster, but were otherwise not that dissimilar from ordinary computers. Very rapidly though supercomputers became parallel systems, i.e. were effectively many computers able to communicate with each other. In effect they were clusters, but special clusters with very high speed communications – but they were effectively just clusters.

Desktops, and some high end laptops, are now able to have one, two, or more, processor chips, but supercomputers comprised hundreds often thousands of processor chips. With the rise of multicore processors over the last decade, desktops, laptops, and even phones have been able to have 2, 4, 8 cores in each processor chip – each core effectively being a processor (this core/processor issue is subtle and too deep for this article, so lets not go into it). Also over the last decade graphics computing has evolved: instead of just being dedicated graphics processors, they have become specialized computers that do graphics very well, but can also be used for some specific forms of general computation (GPGPU). Supercomputers have taken all this on board and are now huge arrays of multicore multiprocessors often with integrated GPGPUs, e.g. Sunway TaihuLight.

What is a Case?

Originally the title for the sprint was "Supercomputer in a briefcase" but then it was realized that briefcases are a bit small and that maybe suitcases should be allowed so as to get a bigger supercomputer. Hence "Supercomputer in a case". Obviously we aren’t going to create a supercomputer like Orac (from Blake’s 7). We are not going to attempt to create a computer able to compete with Blue Gene. In fact, we are not going to create a supercomputer at all. It is just not possible in 2016 to create the computational capability of a modern supercomputer in something as small as a brief case or even a suitcase.

So Why Discuss This?

All modern computers (supercomputers, desktops, laptops, even smartphones) are parallel computers in one way or another, i.e. a computer these days always has more than one processor and/or core. Yet creating parallel programs to solve problems is seen as a specialist thing that only people working with supercomputers do. But if every computer is a parallel computer aren’t we missing something fundamental: supercomputers have just returned to being very big, very expensive versions of ordinary computers. Supercomputers are no longer quite as special in computing terms as they were in the 1980s and 1990: with the advent of multicore, every computer is just a tiny little supercomputer. Think that most people’s smartphones have more computing capability than the 1960’s supercomputers such as Cray-1.

Most people being taught how to program, are taught about sequential programs: programs that do one and only one thing at the same time. Parallelism and concurrency is treated as an advanced topic. Very 1960s.

A note on jargon: In the English language 'parallel' means two lines that never meet, a meaning taken from mathematics; and 'concurrent' means "at the same time". In computing, 'parallel' means "at the same time"; and 'concurrent' labels a way that programs may behave. A book would have to be written, indeed there are many excellent books on the subject, so this article is not the place to delve deeper.

Whilst it is feasible to practically study some aspects of concurrency and parallelism on today’s laptops and workstations, it is very restricted and doesn’t open up the world in the way big computers and especially supercomputers do. So is it possible to create a small machine cheaply that is nonetheless able to allow us to write concurrent and parallel programs to solve problems as we would on a supercomputer: can we create a tiny supercomputer-like thing in a case?

What’s the Gig?

The idea is to enable people to bring together a collection of really cheap computers to create a cluster that is effectively a miniature supercomputer. Can we create a supercomputer by bringing together Raspberry Pis, laptops, and smartphones? There are already 'backplanes' for putting together Raspberry Pis to create a cluster, but this is a specialist bit of hardware. The idea of this sprint is to do something ad hoc. Just connect some computers together with standard networking and, viola, we have a cluster that is a miniature supercomputer. It is about being cheap and easy to assemble and disassemble.

So the idea of the sprint at PyCon UK 2016 is to get people to bring some Raspberry Pis and connect them to a local network and see if we can turn them into a single cluster on which we can run some programs. The idea was first mooted (partly as a joke) in a tweet by Zeth. He and I worked it up a bit using direct messages, and then I took it on as a non-joke project. It has excited the PyCon UK committee, I hope it excites people that come to PyCon UK 2016. I would hate to be in a room on my own with only one Raspberry Pi.

Devoxx UK 2016

Devoxx UK 2016 is just in it’s last half day. The first two days (2016-06-08 and 2016-06-09) were the conference and this third day (2016-06-10) is the half-day workshops. I did my conference session "The Rise and Rise of Dataflow in the Javaverse" in the last conference session slot of the first day in Room A. The slides I used are here. As ever though I did lots of dynamic annotation which is obviously not captured in the PDF. To get a correct "feel" for the session you will have to watch the video. I will annotate this posting when I discover the URL of the video.

The tl;dr of the session is that there is a convergence of functional style, dataflow, and parallelism which means that you have to do it, so there are languages and libraries on the JVM that it behoves people to learn and use.

Update

The video of the session can be found here. It has all the scrawling over the slides as well as the audio.


Copyright © 2017 Russel Winder -