David's Blague

One of my favorite writers is a guy who is currently in the midst of writing a specification for a programming language.  He recently made some changes to the first synopsis of Perl 6, and I am replicating some of it here, because I think it's a useful exhortation.  I hope he doesn't mind:

• The language designer is neither omniscient nor omnipotent, and never will be, despite requests for those particular features. Therefore the design process will be spiral, cooperative, and convergent. The rate of convergence is an emergent property, and cannot be forced, only encouraged. As long as anyone is hacking on any implementation of Perl 6 to make it conform to the test suite, or hacking on the test suite to make it reflect consensus of specification, the rate of convergence will be deemed to be positive. If you are unhappy with the current rate of convergence, please cooperate more with someone else you think is interested in convergence.
• The spec will not be frozen prematurely, but will continue to solidify as various aspects of it are proven (or disproven) in various implementations. Many parts of the spec are already effectively frozen, or are in a slushy state. "The future is already here, it's just unevenly distributed."
• All specced features that have not been proven in an implementation should be considered somewhat conjectural, even if not so marked. As implementations start to agree on what is practical and what is not, do not be surprised if some features that are currently specced may be deferred to future versions; these should still be considered long-term direction in the evolution of Perl 6 over time, and the short-term design should be conservative in preserving that long-term evolution. Note that we are not in a hurry to defer any sections of the spec, even if that would give the illusion of progress. Convergence of specs and implementations will happen naturally as we get implementations that are closer to the spec. It is quite likely that the first practical implementation will largely determine which features are considered to be required in 6.0.0.
• Everyone is allowed to panic once. However, continual panic will be deemed poisonous. Nobody gets special treatment, even if they think special treatment is necessary for success. This means you.

This was added to the synopsis with the log comment: "take another lap around Mt Sinai..." Since it's possible the intended message did not get across, even with this addition, I'd like to remind everyone that the next line in that song is, "'til you've learned your lesson."

Labels: perl, perl6, software

Been reading through the Perl6 specs again.  I like the way the synopses make "promises of parallelizability" on three families: junctions, hyperops, and feeds.  I also like the way these three promises differ from one another.  This post is a little thought experiment, in which I will act as the interpreter for all three.


Junctions
Of the three, junctions are the most difficult for me to grok.  Junctions are a family of four: all(), any(), none(), and one().  One obvious place where this family of four comes in handy:

say "foo" if all(1,2,3) # says foo 

The semantics, in this case, are obvious.  In Boolean context, all evaluates to True if all of the arguments are true, any if any are true, none if (and only if) none are true, and one if exactly one is true.  In this case, it's meant as a nice shorthand for:

say "foo" if (1 && 2 && 3) # says foo

 Except under the covers, it's much different (or at least I suspect it is, inside the brain of Larry Wall).  Since junctions carry a promise of autothreading, the former is a way of handing off to three threads in any order, and the latter (probably) executes in a promised order, (probably) with short-circuiting.  Concentrating on the former, it's not that interesting the way it's written, so let's imagine it a bit differently:

say "foo" if any(bar(1), bar(2), bar(3));

Basically, what I imagine this to be saying is, "go execute bar() in wherever it is convenient, in boolean context, and return the result here.  I will continue whenever either: a) one of you comes back true, or b) all of you come back false."  In fact, you can imagine doing something like this:

do_stuff() if any(breadthfirst($tree), depthfirst($tree));

This could be shorthand for: "do a breadth first and a depth first at the same time, and whenever one of you returns True, the other one can just stop."  It's a very nice kind of syntax in a world where we have a few processors to play around with.

Hyperops
Hyperoperators differ slightly from junctions- they still promise autothreading, but they cannot short-circuit in the same way a junction can.  Hyperops, instead, execute a block-alternatively-statement (this, in Perl6 is called a "blast" I think) on every single member of a list.  It looks like this:

[1,2,3]>>.say # prints 1\n2\n3\n

 Once again, it's a pretty simple example that executes .say on every member of the list.  This, under the covers, might be done in any order, even somewhere else.  As such, any side effects might also occur out-of-order.  I've heard that the order of results is generally preserved, but I'm not at all clear how much that matters in most cases.

Anyhow, back to the previous example, if I did something like [1,2,3]>>.bar(), it would promise to execute bar() three times with three topics (1,2, and 3), side effects would be preserved with all three.  One can imagine how this would be useful for building cellular automata, where one has a grid to keep in synch, and one might do something like @grid>>.next() to yield a subsequent state.

Feeds
Feeds are autothreading just like the previous two, but instead of promising that all will be executed, or short-circuiting, the control of the timing of the autothreading is left to the context (caller?).  A feed looks something like this:

my $a,$b,$c <== 0..Inf

 That <== thing is the feed.  Basically, what this does is says "ask my pointy side how many it wants, grab that many from my blunt end when my pointy end needs them, execute them in any order, anywhere, but give them back to the pointy end in the order they were asked for."  It's not short-circuiting like a junction, it's doesn't guarantee execution on the whole list like a hyperop, but it does provide some pretty nuanced control when combined with either itself or its friends.


I'm still thinking about it, but this increasingly seems to be a nice approach to parallelization, which has good dwimmery (e.g. if you're doing hypers and junctions, even if you weren't trying to autothread, it probably does what you meant), nice fine-grained control (execute all, execute some number, execute with short-circuiting), and still mixes nicely with other parts of the language.

I'm not sure if there are other parallel constructions either already in the spec or on their way in, but I don't find any immediate holes in the semantics here.  Some additional control over how and where the processes are spread around might be achieved with some introspection on the current context, which may require some additional syntax, but that may also be left as an exercise for the reader.

Labels: perl, perl6, software

In celebration of the Federal Government having a snow day today, I tweeted this:

say <. X>[my@i=0 xx 9,1,0 xx 9];map {say <. X>[@i=map {%([^8]>>.fmt("%03b") Z 0,1,1,1,1,0,0,0){@i[($_-1)%19,$_,($_+1)%19].join}},^19]},^9;

It's my geekery for the day.  It is written in Perl 6, and it prints out a Christmas tree, which is also a cellular automaton (Wolfram rule #30).  Thanks to the phenomenal people at Parrot and Rakudo, it looks like this when it is run:


.........X.........
........XXX........
.......XX..X.......
......XX.XXXX......
.....XX..X...X.....
....XX.XXXX.XXX....
...XX..X....X..X...
..XX.XXXX..XXXXXX..
.XX..X...XXX.....X.
XX.XXXX.XX..X...XXX

Labels: christmas, perl, perl6, software

Following a long conversation with a coworker, I wrote down some thoughts about persistent identification schemes (including ARK, DOI, Handle, PURL).  I had the post in the can, ready to go, when it was rudely interrupted by a really interesting presentation, which completely changed my thinking.  I should recap that ill-fated blog post in one sentence before moving on: adding a layer of identifiers doesn't make an existing identifier more persistent, it makes it less so.


Now, that being said, there's a real problem.  If I want to point at something as it exists on a certain date, it's often quite unwieldy to do so.  Maybe I can cache it locally, maybe I can use one of the persistent identifiers mentioned in that first paragraph, or maybe I'm just out of luck.  I point to someone's Geocities site, and it's just fricken gone.  You see, the problem isn't that information moves to a different location, it's that the information at a given location changes.  Or that it disappears entirely.  That's a use case I care about.


Enter Memento.  Herbert Van de Sompel and Michael Nelson gave a presentation about it at the Library of Congress yesterday, and I'm convinced it's a better way to think about persistence.  Basic gist is that you specify a date with a URI, and a combination of clients, servers, proxies, and services try to give you back the thing you were pointing at, rather than the thing that's there now.  I don't love all their terminology or even their implementations, but those are details.  Memento is still a work in progress, and I like the approach.

Labels: libraries, software

Ze Frank has this episode of The Show where he talks about "brain crack." It's the stuff your brain gets addicted to because it could be good. Watch the episode:

http://www.zefrank.com/theshow/archives/2006/07/071106.html

This, Internet, is my brain crack. Been thinking about this for a long time, hoping to write something long. Well, here's something short, instead.

Claude Shannon Invented Computers
Claude Shannon invented computers. Not all at once, and not all by himself, but when you think about the guy who had the "Aha!" moment, it's him. In 1937 (there were, it seems, a whole lot of "Aha!" moments in the thirties), he published his Masters thesis, which showed how Boolean logic and electrical circuitry are the same. One can (and Shannon did) translate back and forth between electrical circuitry and Boolean logic.

To me, that's the "Aha!" moment. He's the guy who made it all possible. In his paper, he designed several electronic circuits, wrote them up in Boolean operations, and became the first guy who "programmed" electronics by manipulating logic. And before the comments start flowing, yeah, I can think of quite a few precursors. Heck, Euclid did the same for manipulating physical space, right? But Shannon's most immediate precursor was George Boole, whose logic system he wrote about.

George Boole Invented Binary Logic
George Boole reduced logic down to True and False. Every statement in his system could always be reduced down to True (a statement that follows from the accepted axioms) and False (one that doesn't). If you took a philosophy class in college and had to make truth tables, it's really Boole's fault.

Boole wasn't the first to take an interest in binary systems: that's an old, old idea. Pingala used a kind of binary notation for describing poetry. Eye of Horus is a binomial system, base two just like binary. Knuth even points out in The Art of Computer Programming that English wine merchants have used a binary system for buying and selling wine for hundreds of years. But George Boole was the one who invented binary logic, and that was one of the necessary pieces for Shannon's thesis.

Leibniz Invented Binary Arithmetic
A century or so before Boole, Leibniz got fascinated with how the I Ching seemed to be organized by a binary mathematical notion: if you make a broken line a zero and a solid line a one, they can be placed in sequential binary order. Gottfried Wilhem von Leibniz went further than that, though. In his Explication de l'Arithmetique Binaire, Leibniz showed how decimal numbers have a one-to-one correspondence with binary. He went on to advocate use of binary, not for the ability to do anything particularly new, but for what it shows about numbers.

Anyone who's reading this should know, I've read Glaser's History of Binary and Other Nondecimal Numeration, so I know he's got a whole chapter called "Before Leibniz." Well, none of it convinced me that I've gotten the wrong guy. Leibniz had probably read Lobkowitz, and probably some of the others who had published, but Leibniz "got" that binary is important, and foundational, and not just another kind of enumeration- it's the one using the least number of digits.

Francis Bacon Invented Binary Encoding
I'm going out on a limb here. Francis Bacon was probably not the first guy, but he did devise an encoding system for English text that only used two characters. He used it as a cipher, but it's not all that different from ASCII that we use now, and for that reason alone he deserves mention here.

Bibbity Bobbity Boo
Put it together, and what have you got? A way to use electricity to do math, encode text, and perform logic operations. Church, Turing, Kleene, and even Gödel all came with complicated logical models for manipulating symbols, but Shannon came up with a simple one, and showed us how to use it with electrical circuits we already had.

Sure, Zuse, Von Neumann, Mauchly, Eckert, they all built machines. But people had been building machines for a long time. Heck, Leibniz had one that could add, subtract, multiply, divide. Ada Lovelace and Charles Babbage had one. Jacquard programmed his fricken looms. But Claude Shannon showed us, hey, just use electricity.

So, if you're here, and you're trying to answer the question, "who invented computers?" my answer is, we're all still doing it. But Claude Shannon, the guy who had that "Aha!" moment in 1937, gets my vote for Parent-of-Digital-Computing.

Labels: computers, software

The Supreme Court of the US just heard oral arguments in an interesting case about business process.  Bilski v. Kappos (warning, it's a PDF) was argued in such a way as to exclude patentability of software, though it has been described as relevant to that conversation.  I read the oral arguments this morning.  I'm not a follower of the court, in general, but I care about this topic, and I had a spare half hour. 

The thing that struck me most profoundly about the proceedings was that the justices are smart.  Really smart.  The most impressive sound bite was from Justice Breyer, who offered the following hypothetical argument (top of page 45 in the transcript):

"...this is not a machine. The machine there is a computer. This is a program that changes switches, and that is a different process for the use of the machine."

That is, perhaps, the best description of software, and the best argument against software patents I have ever encountered.  I'll be interested to see where the court lands on this in the future.

As for the Bilski case, it's too bad both sides can't lose.  Either way, though, the court gets a big thumbs up from me.

Labels: patents, software

Project managers have a single superpower: to cause an incomplete project to be complete, without accomplishing any tasks. This is done by convincing stakeholders they are satisfied with the work that has already been done, and declaring it ready for launch.

But let me back up. Project managers only ever get asked two questions: "Is it done yet?" and as a follow up, "When will it be done?" The entire profession sprung into existence because this question was asked so frequently and ardently it merited a person dedicated only to repeatedly explaining that no, it is not done yet, and here's why.

So on one side, project managers have an irresistible force: curious stakeholders. On the other side, they have an immovable object: programmers. When we do our job well, we keep both sets of people happy, with some help from both sides.

Communication. Let's say a programmer hasn't finished a task, because she got drunk at 10AM on a Tuesday, and passed out on her couch. Huge problem. Unless... let's say that programmer told her project manager the week before that she was planning to get drunk and pass out on her couch on Tuesday, and that the task scheduled for that time was likely to go uncompleted until it could be rescheduled the following Tuesday.

No problem! The project manager will then send an email that says something like, "Due to an unavoidable conflict with another project, a critical task will be delayed for a week, causing one week slippage to the final schedule." And nobody will care, because they are finding out ahead of time, and can, in turn, do whatever it is they do with this kind of information when they receive it.

Which brings up a second: planning.

Project managers function best when everyone else does just a little bit of planning. We don't expect people to plan things like car wrecks or deaths-in-the-family, but we do expect people to plan things like Christmas or hiking the Appalachian Trail. Some discretion is involved, obviously.

With that, I now intend to arbitrarily and capriciously demonstrate the superpower that I explained earlier. This is my last post about project management, because the series is finished. Four posts were planned, and four posts were made. Great fun was had by all. The project was a success.

Yay!

Labels: programming, project management, scheduling, software

A second, and dramatically less useful abstraction The Meeting. Ah, meetings. Meetings are easier to define than tasks, and on some project teams, more frequent. As with tasks, I offer a definition:

meeting (n): a scheduled event, in which attendees follow an agenda

Meetings kind of suck. I think this definition gets that point across. However, there are both social and technical reasons for them. The social reason is that many programmers dislike having to tell a group of people they haven't completed a scheduled task, which is a stupid, sucky reason to have a meeting, but there it is. The technical reason is that meetings are high-bandwidth and low latency, which is sometimes needed.

A few words on how project managers can make meetings suck less... send out an agenda ahead of time. Put items on the agenda, with approximate times next to them, and people's names. If a person's name is on the agenda, attendance in mandatory. Otherwise, it's not. Start meetings on time, and end them early. Socialize afterward. Make announcements via email: announcements do not require low latency or high bandwidth. If there's nothing requiring low latency, and nobody needs to be shamed into completing a task, cancel the meeting. I don't do that nearly often enough.

And a few words to the programmers... don't be a jerk, not even if you really hate being in the meeting. Either come and act like a competent grown up, or show some intestinal fortitude and blow it off. If a programmer's name is on the agenda next to an unresolved item of business, one way to miss the meeting without making the project manager go insane is to resolve the item prior to the meeting, and let everyone know.

And a few words to the stakeholders who will inevitably also be in the meeting... if you want to ask a question or discuss a topic in the meeting, email the project manager and ask to have it on the agenda. The project manager can probably make sure there will be a good answer waiting in the meeting, and may be able to pose it in a way that doesn't piss off the programmers.

Lastly, ten words from Despair, Inc. about meetings that should always be in the back of everyone's mind when deciding whether to have one: none of us is as dumb as all of us.

Labels: programming, project management, scheduling, software

(editorial note: despite my previous post complaining about the term Project Manager, I'm still going to use it. For what it's worth, it makes me feel slightly dirty)

Tasks are one of the most useful abstractions in the project manager's toolkit. Unfortunately, they are also one of the most poorly understood. With the hope of easing some of this confusion, I propose the following definition:

task (n): scheduled work, assigned to a person, for a purpose

In my initial writing of this post, I expounded quite a lot, and realized all my explanations were detracting from, rather than adding to the clarity of that definition. I think I'll save muddying of the waters for subsequent installments.

Labels: programming, project management, scheduling, software

The title "Project Manager" brings to mind an interaction with rms several years back. I was working on a "Content Management System" for a mutual acquaintance, and he asked me (seeming innocent):

"What are you working on?"

"A Content Management System." Simple enough.

"What do you mean?"

"Well, it's a system that manages content." I hadn't known Richard long at the time.

"That's like saying you're 'doing something with stuff.' It doesn't make any sense." Richard had a very good point, but I was still stuck. "What kind of content?"

"Mostly news articles, I guess."

"What are you doing with them?" he continued.

"Uhhh... publishing them to a website?"

"So you're writing a piece of software to publish articles? Why don't you call it an 'article publishing tool'?"

Using the title Project Manager is like saying I'm somebody who does stuff with things. I've spent a while trying to come up with a better alternative, and last night while I was putting the kids to bed, it was stuck in my brain for some reason.

What I do, is schedule. It doesn't sound glamorous, and much of it isn't. I write meeting agendas and send email and set up conference calls. I am tasked with guessing when it's going to be done, for some arbitrary values of "it" and "done". Most of all, I try to figure out the sometimes complex interdependencies between tasks and personalities.

What I schedule, is software. I don't know if scheduling a tank or a road or a rocket launch is really the same. Project Management Professionals [sic] treat them as similar problems, but I don't ever run out of steel or tar or hydrogen fuel, and so far I have never run out of bits, either.

I am a Software Scheduler. Since that job title still sounds like I'm someone who predicts the future and bends the space-time continuum, I hope to make this into an occasional series about how software gets scheduled.

Labels: programming, project management, scheduling, software

A few weeks back, I posted some thoughts about full stack development. The short version: full stack has its uses, but I'm not a huge fan. I inadvertently left commenting enabled on that post, and Jesse Vincent (as if he weren't already busy enough without responding to my FUD) made a thoughtful, cheerful response:

...or if it's not fast enough or pretty enough, the framework can be improved and all sorts of applications can benefit ;)

Good point. And Jifty (Jesse's full-stack system of choice- e.g. he's the author) is changing my mind about a lot of things. Some of my favorite things about Jifty so far:

1. Template::Declare. I've used (and liked) Mason, HTML::Template, Template::Toolkit, and a few others in production, and they all share something in common: they allow you to mix markup and logic. Well, Template::Declare has shockingly ended all that by eliminating the markup from the equation. Eek! I may never write markup again. Hope I can do everything I need to in CSS...
   
2. Jifty::Dispatcher. Dispatching is another difficult problem for which there are a few reasonably good solutions. But I've never used one as intuitive and powerful as Jifty::Dispatcher. Together with Jifty's native ability to do continuations, performing tasks like selectively requiring a login have never been easier.
3. Jifty::Plugin::*. I'm a bigger and bigger fan of mix-ins as my design pattern of choice when creating a web application. Jifty's Plugins are obscenely easy to use (add a line config.yaml), and not as obscenely difficult to write as one might expect. Good plugin architecture is one of the reasons I've used CGI::Application in the past, and I think Jifty made all the same right decisions.

It's not all candy and roses, but I think it's headed that way. Jifty still wants more $hype, more documentation, and fewer bugs, but those things will all come with time and use. In the meantime, it's nifty and getting niftier.

So, Jesse: mea culpa. I think the system is great and getting better.

Labels: development, software

A couple weeks back, Rob Pike revived this weblog, which in turn triggered this discussion (among others, I'm sure). The only post on the site at the time of this writing (other than the short "revival" post) is a long rant about Richard Stallman that Pike wrote in 1991, and re-posted in 2004.

Rob Pike is a towering genius of computing.

And he's dead wrong.

To begin with, Pike's 1991 rant is showing its age. Free Software is about as mainstream and commercial as you can get these days. Hell, Sun Microsystems is releasing about everything they do under the GPL, and that's just for starters. IBM, RedHat, Oracle... all big players in Free Software.

But that aside, Pike intentionally mis-represents Stallman's viewpoints, ostensibly for the sake of making a point:

"Free Software is like Free Love, a hippie pipe dream in which computing is free from venality, commercial interests, even capitalism."

Now, Stallman himself may be a bit of a hippie pipe dream. He is, as far as I can tell, free from venality, commercial interests, and, yes, capitalism. He is the President of a prestigious foundation, yet he makes no salary.

But Free Software?

Come on, Free Software is more like Free Markets than it is like a hippie pipe dream. Pike almost surely knew this when he wrote the rant. And if he didn't know it then, he certainly knew it when he re-posted it in 2004.

Pike's rant goes on to talk about how civilized everyone at the "protest" was. He even cites this as evidence of him being right, and Stallman being wrong. He could just as easily have taken it to show how civil the Free Software movement was, even in 1991. But he didn't.

Software patents do hurt programmers.

I am a programmer.

So they hurt me.

I don't believe they're good for AT&T, but they're definitely not good for programmers. If you don't understand why, read Donald Knuth's explanation. But that hasn't kept me (or Stallman, or the rest of us) from being civil.

I started out by calling Rob Pike a towering genius.

He is. But he's dead wrong on the issue of software patents.

Labels: freedom, software

The talk in San Diego is come and gone, and it was incredible fun. Thanks to all of you who posted materials, emailed me, posted on your blogs, and came to the talk.

I had a great time.

I learned things.

For instance, that the title for my talk, had I used the ablative correctly, would have been Deus Ex Automatis. File that under things I never would have known.

The slides are linked above. The text of the talk, I'll be trickling out in segments here on my online journal. The first segment, this afternoon, will be my "historical" introductions. Lucky for you all, I'm no more historian than I am Latin scholar.

Labels: cellular automata, etech, software, technology

I am a Kool-Aid drinker. One summer, my college roommate and I drank 400 gallons of the pre-sugared variety, and saved all of the containers in a large Kool-Aid pyramid. And not just for the free stuff we could have gotten, had we bothered.

That, as usual, is a tangent.

I also drank the Free Software Kool-Aid. Richard Stallman is right. Open Source misses the point. But I think Richard also misses an important point, which I'd like to address: the development practices associated with "Open Source" are not, in fact, practical or pragmatic.

When I, a programmer, write software, I do so to solve a problem that I have, or that someone else has paid me to have. I do so with the understanding that I will spend the least total number of hours possible on the lifetime of this problem. It is because I understand the virtue of laziness in a programmer.

As soon as the new GNU General Public License is completed, it will become my license of choice. In the meantime, I use the current version (GPL v2). I license the software under these terms, so that other people will have the option of making my software more useful for themselves.

In doing this, I exercise my essential freedoms, and accept the fact that others share these essential freedoms (they are, after all, essential). In exercising my freedoms, I have sacrificed nothing, and possibly provided something of value to someone else. But it is that person's freedom to make my software valuable to herself, not my responsibility.

The latter is a theoretically pragmatic argument (If more people use the software, it will be better) made by advocates of "Open Source" philosophies.

It is, simply put, not entirely true.

The siren song of the programmer is to solve generic problems (e.g. Object Relational Models or Full Stacks or Productivity). And in doing so, programmers often doom nascent efforts to poor performance, long development cycles, and, perhaps worst of all, frustration for the programmer.

Whereas solving specific problems well results in joy for the programmer, code that is written quickly and performs according to the programmer's need or skill, and code that may (or may not) be generally useful.

Over the long term, after enough programmers have done this, meta-programmers (e.g. RedHat or Apache or Ubuntu) can string together enough narrowly useful pieces to make a system that is generally useful. But systems that set out to be generally useful rarely are.

To me, the ethic of Free Software is a true ethic of freedom, and my chief responsibility is not to take these freedoms from anyone else.

Now that's pragmatic.

Labels: development, freedom, software