layer-8

a-splunking we will go

2006-05-18T23:45:00.000+01:00

Splunk 2.0 is just out, and I thought I'd give it a go. I've just finished downloading the thing on to one of our servers. It is now 23:47 and the download took a whole 3 minutes.

I'm doing the "look mom, no wires" installation kind, no manual, no nothing, just to see how it goes.

we start off by chmod a+x splunk-2.0.1-linux-installer.bin , and then ./splunk-2.0.1-linux-installer.bin

Licenses... oh well... let's just say "yes", shall we?

The defaults are pretty sensible for a test install. The only thing I changed was where it asked me how much free space to leave on the device.

It seems to be installed and running, and the whole process took less than 10 minutes, download included.

I'll let you know in the morning how fun it is to use.

The Decaffeinated Penguin - thoughts on Java and Linux

2006-05-18T01:36:00.000+01:00

James Governor commented on Java and Linux, and it stirred up a few thoughts.

I remember only too well how the Java/Linux relationship started. It started when Sun wouldn't make a JRE, let alone a JDK for Linux.

So the Open Source movement, as always, routed around them. Some went on to develop things like Kaffe, an open source Java runtime. But it was always a work-in-progress, sometimes slow, sometimes a bit buggy. But it gave most people the impression, in those earlier days, that Linux wasn't really the platform for Java.

Where does that lead you? Windows. You see, the problem with Solaris is that most developers don't have a Solaris box sitting on their desktop, or under their desk. Solaris boxes are kept by systems administrators, and tend to be used primarily for "enterprisey" projects, which doesn't really include installing a new language and environment and fiddling around with it to see what you can do with it. So the kind of machine the average developer has unfettered access to is, of course, Windows or Linux. Of course today you can run Solaris x86, but not when all this was happening. A couple of years ago Solaris x86 was something of a non-supported product, always rumored to be discontinued with a hardware compatibility list that could fit on the back of a business card.

And since they had no option, developers went on to install Java on Windows, and to develop on Windows, and to deploy on Windows. Sun did that, by merely not supporting Linux early enough.

Now Sun comes talking about Java and Linux. Well, they're a bit late, and they'll have to run a bit to catch the train. Without Linux, the Java community has pretty much settled on Windows ou Unix. And without Java, the Linux community has pretty much settled on whatever other language they choose, Perl, Python, Ruby, PHP, C.

Making developers turn to Java on Linux is probably going to take something more than just hype. You can make it as god as any other platform, it still won't be a reason to switch. You have to make it run better on Linux than it does on Windows, aim for better integration with the kernel API, aim for speed, aim for integration of the Java toolset with other GNU/Linux tools.

Jaime Cardoso was commenting, on an unrelated matter, that the "if you build it, they will come" motto wasn't really true. Actually, I think he's wrong, I still believe that if you build it, they will indeed come, if:

if they aren't diverted into something else that has better promotion than you
if they aren't already somewhere else

Java on Linux suffers from both these problems. The Java developer community isn't all made of fresh-out-of-college junior programmers. Some of the people in the community have been around as long as I have (does anyone still remember Duke?) and were hurt by the no-Java-on-Linux decision Sun made early on. So going back isn't really "the thing we have been hoping for all our lives". We were hurt, and moved on. So we are already somewhere else.

As for the new programmers, Java has a lot to compete against. Windows on one side, with C# and .NET and flashy McDonalds Certified Food Specialist titles that look good to a 19-year-old. Python and PHP and Ruby on the other side, with the save-the-world-in-4-lines-of-code-and-get-a-free-model-train approach.

So where does that really leave Java? What is the competitive advantage of Java nowadays, other than being taught at schools?

The "write once, run anywhere" ideal will not suffice. Portability is nice, but most client Java solutions have problems whenever they touch the graphics subsystem - slow, have to be "tuned" for different platforms, etc., and most server-side solutions aren't just designed to go around running "anywhere". And you can still run PHP or Perl pretty much as "anywhere" that you can run Java - I know, different languages, different features sets, different approaches. But still I don't see programmers flocking to Java and away from Ruby and friends.

Mobile devices come to mind. But where does MIDP come into the Linux equation, if anywhere?

So let me know if I'm wrong, but Sun's current "Java on Linux" drive is just too little, too late.

Double Standards

2006-05-13T02:14:00.000+01:00

In reply to a post on James Governor's blog, I ended up writing what would seem to be a "defense" of Web Services versus more "pragmatic" approaches such as REST. I'm transcribing the full comment here, but allow me to clarify:

I've gone to the point of believing that a bad standard is better than no standard. Choice is good, but I'd like to see convergence just for a couple of years, and then see choice again, and evolution in standards. Right now, I just wish things would interoperate without needing two completely separate interfaces, one to deal with WS-*, another to deal with REST - and sometimes yet a third and a fourth to deal with still some other interfaces.

The post is as follows:

Disclaimer: most of my work life was spent in banking and insurance. YMMV in other industries, I've been involved in some areospace projects, sattelite data collection and such, and in a number of telco gigs, and REST does seem more appropriate in those environments. But today I'm writing from a financial services point of view.

Even for read interfaces there are - more often than not - security requirements. Not everyone can read all data, not all servers can consume all interfaces, some people have access to aggregate figures but not to individual items. And with this need for control - be it real or just perceived by the users, but, after all, they're your customers, most people don't just code for fun - there comes a need to make control centralized and enforcable. It just doesn't make sense to have every single application, every single service, every single datasore implement their own versions of access control, permissions, data visibility, and auditing.

I agree that WS-* is something of a mess. It was designed by commitee, it was design with no regard for practicality, I know all the excuses. But the same problems have existed in all call interfaces that came before, and most of the time it was even worse. Having a bandwagon join behind WS-* would, at least, have the benefit of doing what CORBA could not do, defining a worldwide call-interface standard.

But, as I've said so many times, the "good" thing about standards is that there are always (at least) two (thinks ASCII and EBCDIC, 220 and 110 volts), and human kind has again fulfilled this promise. Even before WS-* was fully deployed and understood, a number of "smart" fellows decided they could always "cut some corners" and came up with REST. Of course it's potentially slower. Of course it's more complicated. If I think about solving only MY problems I can always come up with something faster and easier.

But it doesn't cover a lot of people's needs - needs that have driven WS-* to be - at least partly - as not-simple and not-fast and not-easy as it is. And it's because of those people that REST can not be a universal solution - unless people start grafting other features upon REST, and end up having another WS-*, which they will always claim is better, because it's their WS-*, not someone else's.

Some of us have been around long enough to see all this happen before. Me, i'm still an optimist at heart. I really had hopes that WS-* could be "the one to rule them all". Not that I like it, but I was willing to concede personal liking to standardisation. Apparently, a lot of people didn't agree. Another chance missed - history, again, repeats itself.

Summer Fun

2006-05-02T14:15:00.000+01:00

Too many things, too little time, and a bit of the flu prevented me from posting in the last few days.

In these days, Johathan Schwartz suceeded Scott McNealy as CEO of Sun. While, from a geek point of view, it's a step in the right direction, I'm not sure how it will turn out from a market point of view. It still sounds solid, I mean, it's not like they're turning the company over to Bill Joy or something (not that I don't admire Bill Joy, it's just that there's always a risk capital in putting too much power in the hands of editor frontmen...)

In other news, Apple is interested in porting ZFS to MacOS X. Probably the best move they could ever do in the filesystem arena, combining what is starting to become the world's most interesting operating system with the worlds definitely most interesting filesystem implementation. These are good times to live in.

Yet in other news, we have three totally unrelated events, some of which may interest you in participating:

DARPA is issuing the call for the 3rd Grand Challenge; this time, the robots invade the cities :)

Closer to home, Red Bull (the soft-drink manufacturer) is hosting the 2nd Flug Tag in Lisbon, Portugal. If you can't afford a DARPA robot-car, build a plane and crash it into the sea ;)

If you are not of the building inclination, and would rather sit on a sofa, the Vodafone Best Seat Sofa Race may just be the thing for you. They provide you with the sofa, you just have to talk two friends into pushing you around, and off you go.

There are your options, now don't complain you have nothing fun to do this summer.

Sun, Apple, and the Fortune 500

2006-04-12T12:30:00.000+01:00

Ben Rockwood mentioned that the Fortune 500 list for 2006 is now online, and commented that Sun's position, at #211, is still impressive.

Sun's position is indeed relevant, even slipping from #194 last year, #211 still makes you a very large and very relevant company. Still, it means that Sun is below Apple (which it wasn't in 2005), and profit figures aren't impressive, either, with Sun reporting a 107 M$ loss that places it at #36 in the "Losers" category.

Blame Nick Carr or not, IT, especially hardware and operating systems, is indeed seen by many managers as a commodity of sorts. It's up to the challengers to change that perception - not the incumbents. So HP and Dell can play along with the "commodity" approach and still profit from it, while Sun and Apple can not.

Apple has done a pretty good job at it; it's very easy for the consumer to distinguish from an Apple product and a non-Apple product, they have a different perceived-value, thanks to marketing but also to some true innovation and design.

When it comes to Sun, I talk to a lot of people that really find little subjective difference between Unix boxes, they are ready to choose Sun or IBM or HP based on reasons other that the platform's merit. They are often wrong, but it's not a trivial task to prove them wrong before they make their decision. IBM and HP have a lot of success stories they can come in and tell you. In the end, the decision is based on price, on "synergies" (whatever they may be, real or not) and on "ability to deliver on time".

What is Sun going to do about it? What makes a Sun product better than its competition? And does the average CIO know about that?
Those are the questions that will define Sun's rank in the Fortune 500 2006 list.

windows and unix, oil and water

2006-04-12T02:14:00.000+01:00

Some days go by without even remembering that there are different software platforms and different system philosophies. In this ever-more-so service oriented world, everything quickly becomes a service, and platforms are readily available to do whatever they are best at doing.

It is know that when all you have is a hammer, pretty much everything starts looking like a nail in some way. So the weltanschauung of a distributed systems architect is, often, that everything is a distributed system, even your desktop.

For years, I've had two or three machines as my "desktop". Usually, one of them is a headless linux box under the desk (does that still count as "desktop"?). The other is usually a Windows box that I run Microsoft Project, Visio and sometines Outlook on. The third is a small Solaris box in the development datacenter with a big sticker with my name on it.

Why would an architect need - or want - three boxes for daily usage?
First, you must answer the question of "what does an architect do".

Architects don't spend all their time "architecting" things away. Beneath the architect's skin, there's someone that needs to hack, to explore, to experiment.
Software and systems architecture is a relatively young field, in a fast changing environment. Designing software and systems is not a close parallel to designing houses. In housing, materials and building techniques have developed in the last 20 years, and so have people's requirements,
but those changes pale in comparison with the yearly changes in the software world.
That makes software architecture a partly experimental discipline. The only architects that can do without the hands-on component are the ones that have large teams to do the hands-on bit for them. Still, it's like being a painter without ever touching a brush - and most architects do have the hacker bug in them, so we tend to enjoy the tinkering bit.

Living daily with all the different platforms gives you the true notion of how they can be best combined, so eventually you start looking at them as complements rather than alternatives. Even at home, where we run a Windows box, a couple of Macs (one OS X, one System 9), a FreeBSD and a Debian box, platforms tend to get used for what they do best (we run graphics and audio stuff on the Mac, the Linux box is for development and pen-testing, the Windows box runs OpenOffice, Firefox and a few more things).

But sometimes you have to step outside this confort zone.

Today, I had to setup a way to run remote programs on three dozen windows servers. I tried a number of things, from sysinternals psexec to Windows Scripting. The solution that seemed to be more stable and work best, in the end, was an old friend - ssh.
So I went and installed and configured OpenSSH on 36 Windows boxes in less than an afternoon, complete with public key authentication, and was quite happy with the result.

And then I remembered why I dislike the Windows platform so much.
It has to do with expectations.

An API is an API. A system API, even more so.
The behaviour of a system call is a ponderous thing - it cannot be changed because of a whim, there are whole layers of software that depend on it.

Unix-heritage systems understand this, so they implement system functions with the upmost care, sometimes taking endless time to discuss, to review, to approve a small change. This upsets a lot of people, but changing the API would upset a lot more.
Linux takes a somewhat different approach. The kernel and libraries evolve faster, but there is a serious peer-review process going on in the kernel mailing-lists, and you have well-chosen and responsible people looking over the whole process. So change happens, while disruption is kept at a minimum.

On Windows, they must have a process. Some process. And it's probably good, as processes go. But the outcome sometimes sucks.
Somewhere between Windows 2000 and Windows XP, someone decided to change the behaviour of the RunAs call regarding system-wide resources, such as mapped drives.
In Windows 2000, if a user maps a drive, processes that run under that same account will find the drive mapped and will be able to use it. That allows you to have a session owning the desktop, under user fred, mapping a drive as z:, and then ssh into the box as fred to check on the status of the drive (whether it is still connected or not, if you can write on it, what the free space is).
In Windows XP, you can no longer do that. The RunAs call doesn't export user fred's environment or mapped resources. If you want to check on the drive, you have to map it again - which is not what you need nor what you wanted in the first place.

I'm not even arguing if the decision is good or bad. It doesn't look like it can add much security; if the original user has the power to access a resource, then any other program running as the same user will be able to access it using the exact same credentials. But it does prevent you from inspecting the logged-on user's environment, or to have helper programs be spawned in the background to reconnect the drive (since they are launched by system, and use the RunAs function to change into the user's identity).

The outcome of it all is that I'll have to go find yet another solution for my problem. I'm sure I will find a good one, that's not the part that I'm upset about.

But this close encounter with the things that are wrong in Windows, in the Microsoft change-management process over the operating system, and with the pains of enduring support of a complex software system on top of an operating system that keeps changing for no apparent reason reminded me of why I've chosen Unix over Windows for the last 20 years.

Reliability and People

2006-04-04T01:23:00.000+01:00

If you want a guarantee, buy a toaster.
Clint Eastwood

Game over, time to turn the page (April's fools post)

2006-04-01T23:59:00.000+01:00

This is my 2006 April fools post. None of what is said is actually true, as befits the tradition.

I've had it.

If you read a comment I made yesterday on some blog, you'll know how upset I am at corporate "wisdom", that keeps taking the wrong decisions, over and over and over again.

Today didn't do much to clear my head, on the contrary.

So I've decided it's time to quit.
I am leaving my current position, am no longer going to work in IT, and I've accepted a long-standing proposition to work as a consultant for a major portuguese player in the fishing industry.

And what do I know about fishing, may you ask? Well, a lot more than I do about computers. You throw stuff into the water and you lure fishes to come out of it. With computers, with all the stuff I've thrown into them, I could never get a fish to come out of the drive or anything; a couple of bugs, maybe, but never something as complex and wholesome as a fish.

This decision has been long in the making, and I am aware of all the consequences. I do know I'll suddenly start having a lot more problems with computers, because I will no longer be a computer engineer. The dress code will be an advantage, since I won't have to change much - I can go on wearing high-collar sweaters and sturdy boots, and that's ok by me.

One of the main advantages will be that in the fishing industry it's a lot harder to bring your work home with you, unless you happen to live by the shore (which I don't). Well, you can bring the fishes, but by then they no longer count as "work", the right word would be "dinner".

Lastly, I would like to extend a word of appreciation and thanks to all the people I've worked with in IT during the last 20 years. Your example and inspiration has always been a source of continued strength and motivation for me. It's a pleasure and a privilege to work in an industry with such dedicated and inspired minds as I've had the chance of knowing and working with, on countless occasions.

And if you ever happen to drop by the docks some day, pay me a visit. I'll be the one in the yellow overall, smoking a pipe and making witty comments about the weather.

The thing about Ruby

2006-04-01T04:51:00.000+01:00

It seems that one of the "hot topics" of late in my corner of blogsphere is Ruby.

I first heard of Ruby in 1997. It was being described as a great language for a number of things, especially web apps. Well, everything was supposed to be good for web apps in those days. I remember I was toying around with languages, trying out REBOL, Ruby, Python and others.

REBOL was (and is) almost a DSL, a nice one, too, but still a DSL.
Python, i never did like, (and still don't). You can only trust this much on a language that uses whitespace as a significant token, and not just as a separator.
Ruby, on the other hand, seemed promising. Smalltalk meets Perl.

Perl was always my baby (and still is).
I've done more different things with Perl than many people have done with all the languages they know, combined. And it never let me down; not in reliability, not in elegance, not in power. I've heard complains about the terse syntax more times than I care to remember. But then I realize that the people that complain about Perl are the same ones that complain about algebra, and I understand.

What I don't understand is why we're starting a language war - on a small scale, still a regional conflict - abut Ruby. James McGovern on one side, a lot of people on the other, from Blaine and James Robertson to David Hansson of 37signals (to be expected, of course ;) )

Robert McIlree sums it up pretty well, and there's not much I'd rather say that he didn't already cover.

I've lived by IETF practices for too long to consider doing otherwise. "Rough consensus, running code" has served me well for the last 20 years, and I expect it will continue being a sane mantra.

Morten comments on a LoudThinking article that

The enterprise (...) customers (...) are severely encumbered by legacy because they maintain home grown frameworks from yesteryear rather than having a strategy for moving on. For some reason, they find that they must "defend" their investment rather than leverage common open source frameworks (which are far superior to their home grown stack).

Yes, and the exact same words can be said about Microsoft. Our past stays with us at all times,; still our past is also the ground we stand to take the next step and reach higher, further.

Most enterprises have to maintain something called production systems (look it up on google, if you need); also, most enterprises focus on something called "their business", where IT is just a tool, not a goal in itself. It's so easy to talk about radical change when we have little to lose, isn't it? Now start carrying the weight of supporting production systems, and all-of-a-sudden you're not feeling so light and nimble anymore, and your butterfly-ish tendency to embrace whatever is bright and new and shiny starts to fade away.

And then you start to realize than you do have to keep those 200.000 lines of COBOL on the mainframe, because you just can't take the time, the money and the risk of redesigning your entire core systems because somebody thinks that Ruby or Python are much better and modern languages.

If you come to think of it, the only languages found on those Fortune 200 companies James McGovernor talks about are the ones that have been around long enough to evolve into core systems - COBOL, C, and a few others. Java has made a few incursions into that field - heck, but so has Visual Basic.

So let's all stop judging languages by whether they have been around long enough to have creeped into the core systems, and start looking at what they can do today, and wether or not they can help us solve our problems, today, and help us have less problems tomorrow.

Me, I'm just an old dog; i'll stick to Perl.

in memoriam, Stanislaw Lem, 1921-2006

2006-03-28T00:02:00.000+01:00

We were are all left poorer today. Stanislaw Lem will not write another word, will not open any more new doors, will not hold any more mirrors to our society. But he will still make us think, always make us think.

We knew this was coming. Best known for his 1960s novels, (Solaris, Eden, Memoirs Found in a Bathtub), Dr. Lem had ceased writing novels in 84, only to brake that vow with Eyeblink (2000), maybe his last word of farewell, certainly his last world of hope.

Together with Heinlein, K.Dick, and Vonnegut, Stanislaw Lem alone did more for culture and the elevation of the human mind than most national governments ever achieved.

Deep in a way that some find hard to follow, Stanislaw Lem was also the quintessencial European writer, armed with the wit and sagacity that so often populate his works.

During the last 40 years, he taught us to look in the mirror and laugh at ourselves - as individuals and as a species - as the only way to find perspective in a world that has long ago surpassed our boundaries of understanding.

"In my fourth year I learned to write, but had nothing of great importance to communicate by that means. The first letter I wrote to my father, from Skole, having gone there with my mother, was a terse account of how all by myself I defecated in a country outhouse that had a board with a hole."
Highcastle, 1966 (autobiographical)

Together with Borges and Lovecraft, he created worlds that the human mind grasps - and sometimes fails - to understand, showing us, at the same time, how much we have traversed since we left caves, and still how much road lies before us.

Another European, Antonio Machado, wrote "camiñante, no hay camiño; se hace el camiño al andar". Stanislaw Lem showed us a way by walking it. And he will keep on showing it, wherever he is, for as long as we are here to take the steps that make the road.

Professor, for that, we thank you.

Fat Users

2006-03-24T02:33:00.000Z

Jaime Cardoso talks - once again - about "network computers".

First of all, I don't quite understand the concept behind the "network computer" designation. The last computer I had that wasn't on a network was back in the 1980's.

Besides, when I read "network computer" I tend to think of something that acts as a compute node for the Network, something more like a grid node than anything else. So, Marketing has it all upside-down. The oh-so-modern "network computer" is, in fact, an X terminal.

And those, my friends, I also had in the 1980's.

The current definition of "network computer", from an architect's point of view (which is from where I stand, want it or not), still doesn't differ much from the 1980's X-term. Central processing, central session management, remote display. You can log off from the machine in your office and continue your work, as you had left it, from any other terminal in the building, in the company, in the network.

Hmmm. I could do this in 1989. I still can do it today.
The benefits are obvious, and many people have spoken about them. My question, the real question, is Why didn't it catch on?

And the real reason is Fat Users.
Fat Users don't like thin clients.
Fat Users like to think they're special, that their machine is brighter-bigger-better than the guys next door's.
Fat Users like to think they have a distinctive activity that makes them need special tools, configured in a special way. Than makes them need extra tools, that aren't even company-standard.
Fat Users like to change the colours on their desktop, because it makes them more productive.
Fat Users like to have a special wallpaper on their desktop, because it makes them more motivated.
Fat Users need to sync their phones' agenda with Outlook (or something), because Fat Users have this oh-so-very-important life, and they can't miss a meeting.
Oh, and Fat Users need their own printer, that only they can print to. And it has to be connected to their machine - that's how they know only they can print to it.

Network computers aren't the answer - they aren't even a question.
Getting companies to standartize on a platform and on a configuration, getting users to stop worrying about detail and concentrating on being productive - those are the main issues today, as they were in the 1980's.

Fix that, and the door is open for the return of the X-term.

in the beginning

2006-03-24T01:59:00.000Z

In the beginning was the Tao. The Tao gave birth to Space and Time. Therefore Space and Time are Yin and Yang of programming.

Programmers that do not comprehend the Tao are always running out of time and space for their programs. Programmers that comprehend the Tao always have enough time and space to accomplish their goals.

How could it be otherwise?
The Tao of Programming

Technology is more about people than most of us care to remember or admit. Sometimes it's easier to focus merely on the technical side of an issue than to thoroughly dissect and discover the true implications of it in society and in people's lives, and to judge its merits using a complete, 360º approach.

Why do we do this? Because we can see no further than the tecnological horizon? In most cases, no. We do it for one (ou more) of three reasons:

1. The field of reasons and implications is too wide. If you start considering the social and political implications of the domain name system, you'd be discussing it for years, instead of improving on DNS software. Sometimes we take the technical issue ourselves, and leave politics to the politicians.

2. We choose to believe in something, in a positive way; and we choose to let belief be stronger than factual reasoning - even if the factual reasoning would be enough to prove our point, we choose not to fully prove it, because it would take too much time and effort. Instead, we provide statistics, case studies, examples, anything that proves our point. We plainly choose to believe.

3. We choose to believe in something, in a negative way; either from past events, from observed examples, from other's experience, from our own moral-ethic-aesthetic rules and inclinations, or by mere adherence to a mass-phenomenon, we choose to believe something is bad, should be avoided, or is plainly wrong. We tend to support this view in any way that is more comfortable for us, be it research papers, market studies or anectodal evidence. We simply choose to reject and avoid, which is another form of belief-system.

If we were trying to draw parallels to existing non-technology-related paradigms, we would find a close match in some religious systems.
A large group of agnostics - the ones that just don't care and go about with their lives -, then on one side the do-believers, on the other side the no-believers. If you look at any technology clique that is around, these days, you find pretty much the same model, done with differing degrees of emphasis.

A separate fourth group comprises the "scholars", those who study and analyze, and try to keep an apparently unbiased standing amidst the different sects. Whether they can or actually do maintain this equidistance is, more often than not, a cause for its own debate.

What interests me is the mechanism through which these informal groups light - and keep lit - the fires that fuel them. I am a long time observer - and participant - of the global IT community, while at the same time a student of compared religion; so maybe the parallels are more obvious to me than to most people.

The comparison is obviously not perfect, as no comparisons ever are. Still the resemblances seem too great to not be looked into, so that the lessons - and the risks - learned in one field can be applied to the other.

What would the community think?