XP Wabi Sabi (Refactored)

All requested features delivered. Speculation avoided. Mindful of our tendency toward completeness, necessary code is added, unnecessary code is removed. Refactored. Implementations incomplete - shadows of the their real-world counterparts, yet precisely the functions and properties required. The desire to add more, tempered by the satisfaction of not doing so. Technique and knowledge are increased to decrease their application. Simplicity. I posted that on the WabiSabi page of the c2 wiki on or about October 28th, 2002. I typically feel the same about my old writing and my old code ("what was I thinking?"), but I like this (even if it is a bit pretentious); especially the line about Technique and Knowledge. It cover many of the forces that need to balanced to ship software consistently. Happy Friday.

Posted on: 17 October 2014 | 12:00 am

Test Driving Clojure Macros

When testing functions, one tests: evaluation result side-effects (if any) When testing macros, one may also wish to test: expander code expansion

Posted on: 6 August 2014 | 12:00 am

Immutability and Safety

Work in clojure for any length of time, and you must get used to the idea that data structures are immutable. For programmers coming from imperative languages this can be jarring, (no loop counters? recursion? wtf?) but after a while, you start to get it, then you start to like it, then you start to rely on it - or at least I have. To such an extent that it's jarring not to have them. After a recent javascript coding session, I tweeted: "clojure's immutability has forever spoiled me - destructive operations in other langs feel like bugs now." This prompted Joshua Kerevsky to ask me via email to elaborate, as he has been talking about safety in programming lately. This is a revision of my answer... Clojure1 is safer (in this sense) because there are never any side-effects when working with data. Languages with side-effects on data (i.e. pretty much every other language I've used) require the programmer to keep a mental model of application state and/or adopt defensive programming styles to avoid bugs caused by them. The idea is illustrated by these two examples (I used chrome console and the leiningen repl to run them): javascript: clojure: Javascript arrays are mostly (but not always) manipulated via destructive operations such as sort(), while in clojure, the js array's closest analogue (a vector) is never changed by functions that consume it. It's this "mostly" vs "never" distinction that gives rise to a paranoid feeling that I might be breaking things if I forget something in javascript. I also need to learn more "tricks" to get things to work as I expect. To get the javascript version to behave like the clojure one, we must explicitly copy the array e.g. like this: (bonus: try leaving the var off in front of the concat expression and see how "safe" this version is) One could argue that it is simply bad form to write javascript and expect it to behave like clojure, but entire books have been written to explain to programmers how to avoid side-effect pitfalls in javascript - and the language is almost unusable without them. In clojure, there's much less need2 for this kind of "meta language documentation" - and none for protecting data. It's guaranteed not to change. In the example above, the most likely thing to trip up a programmer new to clojure is the need for doall (leave it out and nothing prints since map is lazy - in the repl you'll need to assign the output to see the difference - e.g. (def foo (listFruits fruits)). This is still a bug, but it's one limited to the function in question, not the entire code base. So my conclusion is that clojure is safer because it has fewer (and much less dangerous) gotchas, the impact of mistakes is limited to the scope of the offending line of code (which will likely be a function or even a let block) and you never3 have to keep a mental model of how state is changing as the instruction pointer advances. It's all right there in front of you. We all make mistakes, but in clojure, mistakes are limited to the context of the function and never due to implicitly mucking about with application state. This adds confidence when making changes, that is simply not there in languages that cannot make such guarantees. [1]Clojure is not the only language that features immutability of course - it just happens to be one I use a lot, and like programming in; nor is js alone in having side-effects; i.e. this isn't about championing clojure (or bashing js) it's about immutability, so feel free to substitute your [least] favorite languages as you see fit. [2] So far at least. Clojure is still young yet, but I don't expect it'll gain this kind of cruft, if for no other reason than because it won't share javascript's experience of being in the front-line of the browser wars. [3] Wanton use of clojure constructs such as ref, atoms & agents can of course lead to such an environment; however even so, clojure provides well-defined protocols for managing change. If the programmer still creates a state-management hell, that's on the coder - as are most problems in coding; no language can enforce safety, only make it easier or harder.

Posted on: 25 May 2013 | 12:00 am

Design goals and Complexity

Programmers solve problems. How they solve each problem is a function of their skill, talent, knowledge & time. The resultant solution will resolve the problem with greater or lesser complexity in the design. Thus, I find thinking about how (and more importantly where) complexity is handled, a useful way to evaluate a design. For this I use the time-honored technique of having three levels: Level Zero (Below this level, the problem is not solved) The programmer has provided a solution. However, the interface is a mine-field; click things in the wrong order, the program crashes. Or it works great, but you need to reread the documentation Every. Single. Time. Or the resultant surrounding workflow is a Rube Goldberg device. However it surfaces, the programmer has placed the complexity on the user. Level One The interface is well thought out and reasonably intuitive. Controls work as expected and the solution is robust - even bulletproof. The solution not only fits the larger context, it improves it. The code itself however, is hard to change (or even understand). It is not well-organized; Or is, but full of tricky, interleaved logic. Or over-engineered, over-patterned etc. Here, the programmer has placed the complexity on the maintainer.[1] Level Two The burden of managing complexity is on the current programmer. The person writing the code takes the time and employs the talent and skill necessary to find an elegant solution that reduces the code to the minimum complexity needed to solve the problem and does so in such a way that is understandable (and changeable) later: The user gains the benefits of a level 1 design, while maintainers are left with clean code to change.[2] Implications Level zero code is common; it's the default for new programmers and an uncomfortable amount of commercial software. The thing of it is: Too often this sort of code is derided (including by me) when there is both a de facto and a cost-benefit rationale for managing complexity this way. Short-term projects are one example; software written for one's own use is another. Level 1 software can be very valuable and enduring. I tend to equate this type of code with the Hacker ethic in all senses of that term. And I don't mean that pejoratively - we all use hacked together solutions every day and much of the world's technology infrastructure is built on it. It is a practical and stable design level. Having the skill (and the time) to write level 2 code is a rare and wonderful thing. Amidst the hyperbole, aspiring to be such a programmer is at the heart of the "Software as Craft" movement and is a worthwhile goal for anyone who aspires to be a professional programmer. I don't believe level 2 is inherently better than a level 1 (or level zero!) - it's about context - however, I think that for regularly edited and changed code (i.e. much IT software), this level of skill is what's implicitly expected (if not gotten) by the customer. Ironically, the time needed is often the first thing that goes as a non-technical customer can only assess code quality based on level 1 considerations and so pushes for faster results because everything "looks" OK. Only later - when their investment can't be changed without major overhaul - do they realize there is something wrong. And so the cycle continues... Summary This model is handy in several situations including: Judging the quality of an actual solution; choosing among different solutions to a given problem; estimating - and even when to stop refactoring. And while I don't believe that all code should be worked until it exhibits level 2 quality, it is what I want others to think of the code I leave behind. And it is what I hope to encounter in theirs. That said, being a successful professional programmer requires (among many other things) the ability to write all three levels of code, and the judgment to know when each is appropriate to use. [1] This may well be the same person who wrote the code. The essential characteristic of a level 1 design is that complexity is put off to future efforts, not the current one. Thus level 1 designs tend to be high in technical debt. [2]How the programmer achieves this is a matter of personal preference and technique. I do not subscribe to the notion that any particular set of programming practices provides this - or inhibit it by their absence. It has always been the people, not their practices that is the essential determinant of quality. Anyone who says different, is selling something.

Posted on: 14 March 2013 | 12:00 am

Expoloring the French Defense (G30 practice game at DRW)

Played an interesting practice game last Friday (1/12) evening with one of my fellow DRW chess teammates, Oliver Gugenheim. After my stupendous blunder last week, I'm interested in playing some practice games - both to drill my pre-move thinking process, and because there's nothing like a bad loss to motivate one to start playing again... Oliver and I wound up exploring a sharp line of the French defense - an opening I have historically not enjoyed playing as white, and so had started learning more about the past week. Oliver (without us discussing it) obliged me by playing a line I had looked at that day so we went a good way into the "book" before (very quickly thereafter) reaching crazy territory. The most interesting bit tho, is actually black's move: 9. ... f6. The conclusion I got from this analysis, is that 9. ... f5 is better (see below for more) and so this was a useful game for this analysis alone... All in all, it was interesting to play, and gave me the opportunity to practice the things above... and it gave Oliver a chance to fend off a ridiculous attack (which is always satisfying if a bit scary at times). Here's the game and my notes (Time Control is G30 with 5 second increment): Event: Site: Round: Date: White: Black: Result: Side to move: Last move:   variations: Next move:   variations: Move comment: And so, QED on this idea. My conclusion: better off building an attack here as White's got the ball. Also, for a bishop sac to have any chance, white really needs another piece. Perhaps once the f-pawn were advanced and White has castled, the possibilty of lifting a rook with tempo might be enough to give the sac some teeth. It'd be interesting to see if I can find any Winawer games with a bishop sac on h7 (if I do, perhaps I'll write a follow-up; regardless, looking at how White attacks here should be fun.)

Posted on: 19 January 2013 | 12:00 am

CICL Chess Match: Board 6 vs Northwestern

I play in the Chicago Industrial Chess League (CICL) for my employer DRW. I generally play one of the lower boards, but I do pretty well. Two nights ago, I played possibly the best chess game of my life... and lost. How? By touching the wrong piece. I had a choice of two recaptures - one that would win the game, and one that would lose. If I had simply moved quickly, that would've been bad enough, but I actually noted the problem with the first recapture, and then did some sanity checks for the other move. Content that all was good, I started my move by picking up the bishop to be captured, AND THEN PICKED UP THE WRONG CAPTURING PIECE. I realized it immediately, but by then it was too late. I resigned a few moves later. The worst bit is I found a strong move on 18. (see below, I'm playing white) - and I saw that position 3 moves earlier. I don't always play this well when I'm rested - let alone after a 12 hour work day... which of course just makes the blunder that much more painful. Moral of the story? When you're winning, when you're tired (or both!). STOP! Check it again. Check each bit, write it down and check it a third time. Then if your sure, make the move. This one burned. Hopefully it stung enough that I won't do it again. Anyway, for your viewing pleasure, here is my wonder-blunder: Event: Site: Round: Date: White: Black: Result: Side to move: Last move:   variations: Next move:   variations: Move comment:

Posted on: 7 January 2013 | 12:00 am

Site Redesign

After an amazing amount of procrastination, I've finally managed to redesign the site. All old links still work (if they don't, please let me know by emailing me at 'bill at this domain name'). Why? Two reasons: 1) I found my old layout limiting for what I want to do and, 2) Something in my toolchain got borked and I wasn't able to update my main page. This last one was a godsend because I decided that - rather than troubleshooting the problem, I'd put that energy toward my redesign. Notable changes: No more MovableType. I got a lot of mileage out of using MT, but in the end I hated having to use a database to store what is essentially static data. Furthermore, the templates and even writing of entries was a bit too much of a flow-mismatch. I wanted to write copy and edit layout in vim and just push the changes to the server. Now I'm using docpad and I have pretty much exactly that workflow My "Thoughts on..." blog is no longer the main page. Right now, the main page is pretty sparse, but as I add more projects (I have some in mind, it'll start to fill up some more. For now, it's an easy way to get at some of the more popular material on my site, and... My dabbling in Fiction. A few years back I started writing flash fiction under my middle name ("Edward"). I haven't in a while, but I hope to start again. So, now that's here too. Total style overhaul. Hope you like it, because I'm sticking with it for a while. My goal was minimalism. This site has always been about what I write. Therefore I want to make it easy to find the stuff here and read it with no distractions. No comments. This may change at some point (I had embedded Disqus but removed it) as I'm going with the Oatmeal Philosophy - I'm struggling against several years of writers block. Feedback can be great, but it can also demotivate. I need to avoid demotivators as I ramp this up. That's it now that I have put everything I could see I needed in place, I'm outta excuses. Whether that means I'll be writing more, we'll have to see...

Posted on: 28 December 2012 | 2:40 pm

Programmer (blood) types

A lot gets said about evaluating technical skills when building teams, but I find that to be relatively straightforward - what's trickier is understanding how a person prefers to work and so how much effort (i.e. change) will be needed for the team to gel once that person is added. Consequently, I spend a lot more time thinking about this than any specific skill a person might have. It's as if each of us has a personality blood-type. Just as transfusing blood among individuals without taking account of their body chemistry can result in serious health issues, so too mixing otherwise competent individuals without regard to group chemistry can greatly reduce everyone's effectiveness. This is true, even if these same individuals would be very effective in different groups. Like one's actual blood type, there are multiple factors involved, but I find three especially important. So, to run with the metaphor let's call them 'A' factor, 'B' factor and 'Rh' factor: A Factor: Knowledge Sensitivity: A person has this trait if I expect the effort needed for them to understand the Situational Context of the team will be a significant factor in the gelling process. Considerations include: current knowledge, learning preferences (online research, training, books, etc), learning-speed, learning-skill, open-mindedness and how (and how easily) they'll learn from their teammates. B Factor: Environmental Sensitivity [1]. This stuff matters to most everyone, but some are more sensitive to it than others. If it looks like it'll be a prominent part of the gelling process, they have this trait. AB: Person has both traits. O: Person has neither. I don't expect the person will require radical shifts in team cohesion to gel with the group (though they still may cause such shifts over time). Rh Factor (+ or -): Tendency toward taking social initiative[2]. This one's hard to define. To get a sense of what I mean, the next time you're in a casual group discussion (ideally of peers), note the people who tend to pick the topics of conversations, suggest lunch destinations and so on. These people have this trait. When work topics arise, they're the ones proposing new ideas and things that need focusing on in the first place. Incidentally, if you believe you have this trait, you might need to reign it in to get a sense of who else has it (and a good habit for you to develop regardless). I see these factors as neither strengths nor weaknesses. Nor as (necessarily) fixed traits. A successful team may contain (or require) people exhibiting any combination of these traits. Adding people who learn quickly or who care a lot about how customers are acting toward the group or are passionate about their work environment [3] is just as important as having deeply experienced people who are agnostic to the details of the work environment. Even when people's quirks or lack of knowledge might seem to be burdensome, our work is a creative activity not a mechanical one. Taking the effort to incorporate people who have a particular genius or ability into my teams has paid off more often than it's been a liability. It is also important to understand this is my thinking process after it becomes clear the person is a plausible fit for the team. That means they've already passed the no assholes rule, otherwise appear to have the skills the group needs and I believe will get along reasonably well with the rest of the team outside the context of the goal at hand. Of course, I also consider many other non-skill traits to be important - e.g.: risk aversion; time flexibility; tolerance for ambiguity, tool/practice prejudices and social neediness are all relevant - but regardless of which ones dominate a particular gelling process, what I'm always after is a better understanding of how the person will complement the rest of the group. Most importantly, reflecting on why people work well together helps me to do my job better. Hope you find it useful for that too. [1] More precisely: How sensitive the individual is to forces acting within or upon the group; this important topic will get a separate blog entry hopefully soon [2] Leadership in the narrow sense of desire to influence direction of a group of people. [3] Tool and process preferences is an entire - large - subcategory that can even be considered an entire trait all it's own. We do so love our tool and practice religions!

Posted on: 10 November 2012 | 2:10 pm

Aretê

"I am not a good manager." I say this to people often, and most of them think I am joking because... well, because people don't say things like that about themselves if they're true and also, I have cultivated some skill in building strong teams that accomplish what they are asked to do. [1]. But I am serious when I say that. for three reasons: I really have to work hard to do the organizational parts of a manager's job (the reports, the budgeting, the note taking, the meetings, the scheduling, the selling of initiatives, etc)... and I don't enjoy any of it. I don't do well at optimizing for efficiency. I don't have much use for roles, particularly ones that tend to specialize activities and artificially segment the work needed to solve the problem (and the skill set of those seeking to do so). Like most programmers, I suck at estimates, I am motivated first by a need to solve interesting problems, and only secondarily at reaping the benefits of doing so, and I have zero desire to "be in charge of others." I also hate process for process sake and generally piss off any project managers foolish enough to work with me (though I am good friends with several). So, how do I build teams and software? By treating efficiency - and even the primary goal of the team - as a secondary effect, and optimizing instead for... for what? Well, until recently I had been (in my head, because I didn't feel too comfortable saying this out loud) using the word: "happiness." Of the members of the team, of my boss, my employer, our customers and (importantly, but until recently neglectedly) me. Make all of these folks happy, and everything just works. Uncomfortable because this is a tough sell to accounting-type folks - and anyone who prepares budgets. "Naive", "Crazy" and "Ridiculous" are what I expect to hear. And the reason I expect to hear it is because it seems really risky - even terrifying - to them when I say anything that implies I'm not thinking about cost and value and ROI, and all those other business terms. But I am. I'm thinking about them all the time. I just don't agree with them on how to optimize them. Also, "happiness" is not quite right: some people are quite happy to do nothing, others are only happy when they are padding their egos at the expense of others, and a whole lot of other types of "happiness" that I don't optimize for. No, it's a specific kind of happiness - especially inside the team - that I am trying to maximize. Joy of doing one's best, professionalism, craftsmanship, cultivating flow, the need for slack. All dancing around it. All not quite it... I recently mentioned here (and on twitter and facebook) that I'm re-reading Zen & the Art of Motorcycle Maintenance by Robert Pirsig. And it's blowing me away. Again. I read it a really long time ago - so long ago, that I had forgotten all of the details and only remembered: "That book really moved me and shaped my thinking." What's been blowing me away is realizing how much it has done that. And so, I've been expecting for days now to find in it somewhere something that supported my view since Pirsig's focus on Quality and Care (not to mention technology) are very similar to my feelings of "optimize for happiness." But it still hadn't felt quite right yet... ...until, today. And I found it: The Greek word: Aretê (translated as "virtue" or "excellence") is a central part of any course on Greek philosophy and I had several classes in college where it was discussed. I was waiting for it to come up in the book (it doesn't until chapter 29) and when it did, I realized I was getting closer. Then, this quote: "Aretê implies a respect for the wholeness or oneness of life, and a consequent dislike of specialization. It implies a contempt for efficiency - or rather a much higher idea of efficiency, an efficiency which exists not in one department of life but in life itself." ~ Pirsig, Robert M. (2009-04-10). Zen and the Art of Motorcycle Maintenance (p. 360). Harper Collins, Inc.. Kindle Edition. And like a key fitting a lock, there it was: I believe that teams (organizations!) should be optimized for Aretê; that teams should be staffed with those who - like Hector the tragic hero of ancient Troy - seek excellence (in their work and in achieving the team's goals) for it's own sake and are not happy unless they are free to pursue it; and finally that the team's success (which will still be a function of external perception of value) will be a natural outgrowth of this process and any attempts to shortcut it (e.g. in the name of efficiency) will actually serve to reduce the team's effectiveness. With this compass in hand, I can see now that what at times appeared to be random objections to process changes and my novel (some would say crazy) alternatives and experiments over the years have really been about trying to keep everyone focused on maximizing the ability to pursue excellence. So, now I'm saying it: If you want to build great teams who reliably ship results: Don't optimize for efficiency, optimize for the pursuit of Aretê. [1] Those who've worked with me on those teams can attest, I've done so by getting the team building part - and my role in it - wrong a lot, but learning from it.

Posted on: 23 May 2012 | 8:50 pm

Logic Engineers

I don't like any of the names for people who create technology solutions for use in the real world. In no particular order: Developer - Sorry, that's for photographic film (and who uses that anymore?) not software - my second least-favorite on this list. Programmer - probably the one I use the most, but the mechanistic connotations: a glorified typist of code; and the implication is that someone else does the thinking, bothers many - leading to... Architect - Horrible. Borrowed from physical construction, it brings in notions of rigidity of thinking and solution; my least favorite, often used by those who use 'Developer' to describe "those who do the thinking around here." Coder - Probably my favorite; however implies that software is the focus instead of the problem to be solved. Hacker - I don't mind this one, but it feels forced sometimes and of course carries negative connotations due to it's use in a security context; It does however, put the problem solving front-and-center and I like that; Software Engineer - Part of my current title; pros: sounds professional and captures the applied science nature of the work; cons: again the implication is writing code, not general problem solving and often - though not in my case - is indicative of a very rigid, bureaucratic culture Software Craftsman - En Vogue with those who see (as I do) that there is more to this work than just slinging code, I have several problems with it: software focus again, the lack of gender neutrality, and a certain pretentiousness (that might just be because some people sound that way when they talk about the movement - which I am positively disposed to, but not a part of) Technologist - This one is popular with the start-up crowd; This is one of the few on this list that recognizes the holistic integrated nature of the work from the solution side; on the other hand, lacks action, professionalism and anything related to the problem (not a coincidence that the startup world likes it, because those that use it are often seeing the project as separating problem and solution too) Sys Admin, DBA, network engineer, etc, etc - I.e. every other technology-related role that isn't primarily about writing software. This is my biggest concern of all as I believe this distinction (writing software and everything else) is horribly misguided, artificial and needs to go (that's another entry) Am I just being pedantic? I'd say yes, except that I believe these terms all devolve from not seeing the holistic nature of solving problems with programmable machines and since I do see it that way, I want a term that encompasses all of it. Thus Logic Engineer. Physicists have Mechanical Engineers to apply their ideas, why don't Logicians have Logic Engineers? Currently, we conflate this practical science with Computer Science which is also a pure science - or would be if we stopped doing that. I know that Engineer has professional accreditation connotations in many disciplines - and I'm mixed on what that would mean given the current state of technological practice today - but I believe this most closely maps to what we are doing: Logician: figure out the rules and limits of valid reasoning. Logic Engineer: figure out the rules and limits of applying logic to real-world problems. We are Logic Engineers. Further, if we stopped slicing the world up into hardware and software practitioners, and started seeing that software is just liquid hardware (and vice versa); maybe we'd have something worth accrediting (though I don't think for a long while yet). So, I'm gonna try it for a while... at least in my head.

Posted on: 17 May 2012 | 6:55 am

Dedication

I've been rereading Zen & the Art of Motorcycle Maintenance. This quote really struck me: "You are never dedicated to something you have complete confidence in. No one is fanatically shouting that the sun is going to rise tomorrow. They know it’s going to rise tomorrow. When people are fanatically dedicated to political or religious faiths or any other kinds of dogmas or goals, it’s always because these dogmas or goals are in doubt." ~ Pirsig, Robert M. (2009-04-10). Zen and the Art of Motorcycle Maintenance (p. 140). Made me think of all the zealotry accusations levied at Extreme Programming practitioners about 10 years back... still seen in reaction to some of the more ardent claims of the primacy of TDD even today. No conclusions, just suspect I will see the next fanatic a little differently next time I encounter one.

Posted on: 11 May 2012 | 10:10 am

Why Teams

Jeremy Lightsmith recently asked on twitter: "Why are teams important? I'm looking for an inspired answer." I don't know about inspired, but here's my thoughts on the topic: First, I believe teams (true teams, mind you, not simply groups of individuals under single managers) are capable of delivering better results than individuals - this is the economic (and in a business sense the only) justification for teams. If they can't outperform, they probably shouldn't exist. Much of my thinking on why they can outperform has been deeply influenced by the book: "The Wisdom of Teams" and I won't rehash it here except to say that I believe the key is in the notion of "complimentary skills" - teams are more efficient because rather than having to staff a group of super-humans (you know the kind that typical HR review material seems to want to turn us all into) you can look for how one person's personality and skills work to reinforce and amplify the others. And that brings me to my most compelling belief of all: It's in our nature. We are social creatures, we need to connect with others, to bond with them, to feel we are working together. Rather than spend our time lamenting our "shortcomings" and worrying about our individual performance (and rewards), forming solid teams means using all of our actual skills as they are to compliment our peers and deliver more than any of us can alone. People simply work better with a group of people who complete them, whom they feel a deep connection to, whom they trust whom they admire and feel respected by. When you have that, people stop thinking about what's wrong with them, and start thinking about how they can help each other to succeed. And when you have all of that, you have a force to be reckoned with. Is this easy to accomplish? No - and you need to accomplish all of that (respect, complimentary skills, trust, common goals, acceptance of each other as we are, etc) to achieve a true team. But I am convinced that the payoff for everyone - from the employer to the individuals involved is so worth it, that I won't willingly work any other way - and I reject any "best practices" or conventional wisdom that interferes with achieving that sense of team in the groups I work with.

Posted on: 5 February 2012 | 2:41 pm

Python Challenge answers 0 thru 4... in clojure

The Python Challenge is a nifty site that presents you with a series of puzzles that it asks you to solve using python; getting each answer allows you to move on to the next puzzle. Python is a cool language and it's a good tool for this job1 However, I'm learning clojure right now, so I thought it would be fun to try and solve a few of them in clojure. Here's my answers for challenges 0 thru 4 (warning: if you want to do these puzzles yourself, reading further now might ruin the fun) Challenge #0 (the "Warmup") Asks you to solve 2 to the 38th power: (clojure.contrib.math/expt 2 38) i.e. just use the exponent function in clojure contrib. Challenge #1 This one throws some scrambled text at you and a clue on what the key is (ROT 2): (defn translate [text] (let [lookup (vec (map char (range 97 123)))] (letfn [(letter? [c] (and (>= (int c) 97) ( Create a lookup table of the chars, a predicate to test if a char is a letter. & a function to get the index of 2nd to next letter (the index loops, essentially making lookup as a ring buffer), then map across the given text, shifting by 2 if its a letter or just returning the char if its not. Challenge #2 This one throws a big hunk of random data at you and suggests you pick out the 'rare' characters: (defn filter-file [path] (let [fs (line-seq (clojure.contrib.io/reader path)) lookup (set (map char (range 97 123)))] (apply str (mapcat #(filter lookup %) fs)))) A quick visual scan of the text led me to a strong hunch the "rare"2 characters were lowercase alpha, so: Re-use our lookup table from the last challenge; this time make it a set, then use the set to filter each line of the file denoted by 'path' (I first saved the text to a file to make it easier to work with); use mapcat to flatten the lines out (this has the effect of stripping empty lines altogether); apply str to the resulting sequence to get the answer. Challenge #3 This one's a big hunk of text too, so a quick refactoring of our last solution results in a more abstract (and higher-order) function that takes a filter function as an additional parameter: (defn filter-file [filter-fn path] (apply str (mapcat filter-fn (line-seq (io/reader path))))) the filter from challenge #2 thus becomes an argument; partial works nicely here: (filter-file (partial filter (set (map char (range 97 123)))) "path/to/file") Now we can make a new filter for challenge #3. This one will need to find character patterns that look like this: ABCxDEF. We'll need grab x. This one just screamed regex at me, so here's a filter that gives us the answer: #(second (re-find #"[^A-Z][A-Z]{3}([a-z])[A-Z]{3}[^A-Z]" %))) An anonymous function3 that uses re-find to match: "not-cap followed by 3 CAPS followed by not-cap followed by 3 CAPS followed by not-cap"; the second element of the resulting vector (because we use parens to create a group) produces x; mapcat et al do the rest. Two big assumptions/limitations here: assumes each target is on its own line, and that the target pattern wasn't on the beginning or end of the line (which was good enough to get the answer). Challenge #4 This challenge requires one to follow a url call chain, passing a different number as the argument to a 'nothing' parameter each time. The resulting page text provides the next number to follow (and/or some noise to keep you on your toes) until eventually we get the answer. This one gets kinda ugly. This is the kind of problem scripting languages are made for (e.g. perl, python & ruby coders would all make short work of this problem). Still, it's possible to write procedural code in clojure, and it's still reasonably straightforward. One decision I had to make is how to GET the url's - my weapon of choice for this sort of thing is clj-http: (require '[clj-http.client :as client]) (require '[clojure.contrib.string :as string] (defn follow-chain [base-url number] (let [result (:body (client/get (str base-url number))) idx (.indexOf result "and the next")] (cond (re-find #"^Yes" result) (do (println result) (follow-chain base-url (/ (Integer/parseInt number) 2))) (= -1 idx) result :else (let [result-vec (string/split (subs result idx) #" ") next-number (last result-vec)] (println result) (recur base-url next-number))))) Take the url as a base & the first number to follow; use client-http/get to grab the page; extract the body of the page; get the index of the phrase "and the next" using the java "indexOf" method - we'll use the index later to parse out the end of the text and get the next number... ...unless of course, we get text that tells us something else (like a message saying "Yes" and then instructing us to divide the last number by two and continue on as before) so... ...we set up a switch using the cond macro: If the result starts with "Yes" make a recursive call dividing the last number by two; if indexOf otherwise came up empty, that's our answer, so return it; else pick the next number out of the result by splitting the end of the string into a vector (using clojure.contrib.string/split) and recur (tail recursively call the function again). The println's could be removed, although they were essential when figuring out what the code needed to do. Conclusion This was a fun exercise; clojure's holding up pretty well so far, though clojure would not be my weapon of choice for that last one; if I choose to do the next five, I'll post them in a future article. Footnotes [1] It's also the darling of the hipster crowd right now -- in many cases the same people who snubbed python when ruby was the hip language about a decade ago... python abides. [2] The official challenge answers also tackle ways to deduce "rare"; knock yourself out [3] #() defines a function where % %2 etc represent positional parameters; the (fn [arg]) syntax would work here too

Posted on: 19 October 2011 | 8:48 pm

My first clojure macro

I'm finally experimenting with writing macros in clojure. Learning macros is (for me at least) a 4 stage process: Learn to use them (pretty straightforward) Learn to read their implementations (including the quoting) Learning to write them (in progress) Learning when to write them (in progress) Those last two are iterative; #4 is especially tricky -- the web is full of general considerations ("when a function won't do", "when you want new syntax", "when you need to make decisions at compile time", etc) - but actually making that judgment in practice, takes... well practice. Hence this exercise. Anyway to the code: Clojure offers the if-let and when-let macros that allow you to combine a let block with testing the binding for nil: (when-let [a (some-fn)] (do-something-with a)) (if-let [a (some-fn)] (do-something-with a) (otherwise-fn)) I found myself (on some real code) wanting to be able to do something similar with try: (try-let [a (some-potentially-exceptional-fn)] (do-something-with a)) (try-let [a (some-potentially-exceptional-fn)] (do-something-with a) ArithmeticException ((println (.getMessage e)) 42) :else (do-something-by-default-fn) :finally (println "always")) etc. So I wrote this (non-hygenic) macro that seems to do the job: (defmacro try-let [let-expr good-expr & {fin-expr :finally else-expr :else :as handlers}] (letfn [(wrap [arg] (if (seq? arg) arg (list arg)))] `(try (let ~let-expr ~good-expr) ~@(map #(apply list 'catch (key %) 'e (wrap (val %))) (dissoc handlers :finally :else)) ~(if else-expr `(catch Exception ~'e ~else-expr)) (finally ~(if fin-expr `~fin-expr ()))))) Thing is... I don't if it's a good idea or not. For one thing its not hygienic (it implicitly declares e that can be used in the handler clauses) though this seems the kind of case that sort of thing is for. For another... I don't know if its correct. It seems to be (I've tested all the scenarios I can think of), but this is kinda like security -- I suspect anyone can write a macro that they themselves can't break, but that doesn't mean its correct. Some things to note: - e is available to handler expressions- the local function wrap allows for a complex expression or single value to be spliced in- any number of handlers can be included- ':else' (default) handler and ':finally' handlers are optional (as are any others!) In short: I'm interested in any opinions/feedback that aim at learning steps 3 & 4 (writing and when to write). Fire away!

Posted on: 12 October 2011 | 8:23 pm

Where Style Rules Come From

From a larger tutorial on Common Lisp Programming Style, comes a nice list written by Peter Norvig & Kent Pitman surveying "where your 'Style Rules' come from": Religion, Good vs. Evil "This way is better." Philosophy "This is consistent with other things." Robustness, Liability, Safety, Ethics "I'll put in redundant checks to avoid something horrible." Legality "Our lawyers say do it this way." Personality, Opinion "I like it this way." Compatibility "Another tool expects this way." Portability "Other compilers prefer this way." Cooperation, Convention "It has to be done some uniform way, so we agreed on this one." Habit, Tradition "We've always done it this way." Ability "My programmers aren't sophisticated enough." Memory "Knowing how I would do it means I don't have to remember how I did do it." Superstition "I'm scared to do it differently." Practicality "This makes other things easier." ( via Common Lisp Tips )

Posted on: 8 October 2011 | 8:59 am

Stand-ups

I've been saying this for a while, now I'm just gonna write it down: Stand-ups suck. Treat them as a last-resort. The classic stand-up: each day a group of people stand huddled in a circle chanting: "His Name is Robert Paulson... His Name is Robert Paulson... His Name..." - no wait that's not it... They stand in a circle bored out of their skulls while each in turn says what they did yesterday and what they are going to do today. A couple of people sorta care what everyone says (usually project managers, or customer reps) everyone else is interested in what maybe one or two people say. No one can go into enough detail to really educate anyone. Everyone's glad when its over. Sometimes people attend many of these per day (cross-team stand-ups anyone?). A few might even make it a significant chunk of their day & job descriptions; I think of these people as "Meeting Moths" - attracted to meetings like moths to the flame. STOP. DOING. THIS. Instead: Figure out who needs status and how often; GO AND GIVE IT TO THEM. It's a good bet they don't need to hear the minutiae of what code was written anyway & you're probably addressing that in existing stand-ups by making sure the programmers don't get too detailed killing any value they might've gotten from talking about what they are doing - dysfunctions beget dysfunctions... Those who are doing actual tasks should be free to talk about them when they need to. In my experience this part of the stand-up gets short shrift when its scheduled and attended by those not doing the work. GET THEM OUT OF THERE. If you do, you might just see spontaneous, short discussions start happening daily among task completors (that's a real stand-up btw, just don't call it that or you'll ruin it by attracting meeting moths) Move people next to each other who need to talk a lot. Conversely, if you already are around each other all day, you probably already know who did what and whose doing what (if not WORK ON COLLABORATING MORE) If after you are doing all of these things, you still feel like you need a periodic daily meeting, then think for a while! Be creative! Don't just do what some book or consultant said to do, use your brain and solve the problem to everyone's benefit as best you can. Only then, if you still feel you need a stand-up, then fine; go ahead and schedule one for the things you still didn't address (but don't cover things already addressed!). The result is much more likely to be useful for everyone involved. All periodic meetings including stand-ups evolve into wasteful time-sucks; They develop an inertia that supersedes whatever utility they initially might have had. Aggressively question their very existence; make the burden of proof be on keeping them, not ending them. Simpler still, kill em all and see what's not being done and then be creative on how to fix it. Who knows you might just get away with one less meeting in your day.

Posted on: 24 September 2011 | 3:07 pm