Owen's Solidsoft Blog

On the nonlinearity of distributed systems

2007-08-20T12:34:00Z

I came across a Wikipedia page on the Fallacies of Distributed Computing. Read these and tell me if they ring any bells:

The network is reliable.
Latency is zero.
Bandwidth is infinite.
The network is secure.
Topology doesn’t change.
There is one administrator.
Transport cost is zero.
The network is homogeneous.

Great aren’t they? If you’ve worked in IT for any measuable amount of time, you’re bound to has assumed, or seen someone assume, one of the fallacies above. But they will always come back and haunt you!

Nigel, one of my colleagues, is a firm believer that things were better in the mainframe days. Things were simpler, more controlled and infintely more reliable. His favourite anecdote recounts a speaker at a conference who went up on stage bouncing a basketball. He compared this to a mainframe, easy to control. He then pulls out a bucket of ping pong balls, throws it into the audience and then asks them how then can control that. It think that comparison is a bit harsh, however, it does outline the non-linearity that distributed computing suffers from when it comes to issues of control, manageability and fault control.

Personally, I think it’s a matter of market/technology granularity. Yes, a mainframe may seem more dependable, but that’s mainly because you’re dealing with one supplier who gets called in when there’s an issue. The end-user never sees the individual components that make up a system, it’s a black-box that someone else manages, so yes, it’s surely going to look like an easier environment. Distributed apps are likely to be running on disperate hardware, across multiple operating systems, supplier and supported by different vendors, all with their own support procedures. And this is where non-linearity steps in. In mathematics, a nonlinear system is one whose behavior can’t be expressed as a sum of the behaviors of its parts (or of their multiples.) (more on Wikipedia). The challenge software/system/technical architects today face is expressing this nonlinearity in terms of risk so that the business can understand it. It still is a thorny issue though.

Do you come across these problems in your day job?

Previously posted on my business/technology blog.

WiTricity: Wireless Electricity

2007-06-08T13:36:00Z

Here’s the coolest thing I’ve heard in a long time. According to an article on the BBC, US researchers are working on a system to transfer power without the need of wires. WiTricity (as this new system has been coined) using the principle of low-frequency electromagnetic resonance to transfer energy from a charging device to one that needs power. And because the resonance is predominantly magnetic, it’s perfectly safe to organic tissue and can be deployed anywhere it is needed. The technology is still in it’s infancy, so you wouldn’t use it to power your laptop yet, however the demonstration the BBC article talks about proves it’s a viable alternative to wired power.

This is an interesting idea following on from my post yesterday about iPhonomics, a concept that states our use of technology is bounded by the power capabilities of the device we are using. If wireless energy is possible, I predict that after the initial pain and cost of implementing this, charging your devices will become ubiquitous and one day, people will still thinking about it; in the same way we stop thinking about how strong our mobile signal is. We’ll just expect it to be there.

(Previously posted on my business/techology blog)

Security vs Access

2007-03-23T10:19:00Z

It’s interesting to see how the right sort of tension in the workplace can bring about better results in what a team is trying to achieve. I’m working on a long term project at the moment for the Isle of Man Government, which is tasked with putting a number of services online for citizens to use. This implies access to a number of back-end systems which normally would only be accessible to civil servants on the Government network.

We have built an infrastructure to deliver these services, however, in order to access real-time data on back-end systems we need to enable communication from the Government’s website all the way to the relevant back-end system. And this has made us extremely unpopular with some of the security team.

So, where are the security bods coming from? Well, one of their fundamental principles is that there is a security perimeter around the organisation and NOTHING should be able to traverse this unless it is initiated from inside the organisation. It’s a valid principle, however following this principle to the letter would mean we could not provide any online services without staging data outside the internal network. Doing this would have had impossible for us, not only because of the volume of data we were talking about, and but primarily because we’re not simply providing access to data, but actually integrating the citizen into a business process that’s running on Government systems.

So were the security team an obstacle for us? I wouldn’t say so, though I might say they were a challenge to accommodate. The beauty of the relationship though was that they questioned us every step of the way and in because of this we produced a more robust, elegant and secure solution. Exposing core systems to the Internet is not something that should be taken lightly and the security team made sure we didn’t take the easy way out. So thank you guys, together we’ve built a better solution for everyone.

(Previously posted here)

Business-speak

2007-02-02T20:08:27Z

The weekend is here, so time for some fun. Here's some essential new Vocabulary for your company-speak database...

1. BLAMESTORMING: Sitting around in a group, discussing why a deadline was missed or a project failed, and who was responsible.

2. SEAGULL MANAGER: A manager, who flies in, makes a lot of noise, craps on everything, and then leaves.

3. ASSMOSIS: The process by which some people seem to absorb success and advancement by kissing up to the boss rather than working hard.

4. SALMON DAY: The experience of spending an entire day swimming upstream only to get screwed and die in the end.

5. CUBE FARM: An office filled with cubicles.

6. PRAIRIE DOGGING: When someone yells or drops something loudly in a cube farm, and people's heads pop up over the walls to see what's going on.

7. MOUSE POTATO: The on-line, wired generation's answer to the couch potato.

8. SITCOMS: Single Income, Two Children, Oppressive Mortgage. What Yuppies get into when they have children and one of them stops working to stay home with the kids.

9. STRESS PUPPY: A person who seems to thrive on being stressed out and whiny.

10. SWIPEOUT: An ATM or credit card that has been rendered useless because magnetic strip is worn away from extensive use.

11. XEROX SUBSIDY: Euphemism for swiping free photocopies from one's workplace.

12. IRRITAINMENT: Entertainment and media spectacles that are annoying, but you find yourself unable to stop watching them.

13. PERCUSSIVE MAINTENANCE: The fine art of whacking the crap out of an electronic device to get it to work again 14. ADMINISPHERE: The rarefied organizational layers beginning just above the rank and file. Decisions that fall from the adminisphere are often profoundly inappropriate or irrelevant to the problems they were designed to solve.

15. 404: Someone who's clueless. From the World Wide Web error Message "404 Not Found," meaning that the requested site could not be located.

16. GENERICA: Features of the American landscape that are exactly the same no matter where one is, such as fast food joints, strip malls, and subdivisions.

17. OHNOSECOND: That minuscule fraction of time in which you realize that you've just made a BIG mistake. (Like after hitting send on an email by mistake).

18. CROP DUSTING: Surreptitiously passing gas while passing through a Cube Farm.

HBR - Breakthrough ideas for 2007

2007-02-01T12:29:26Z

Harvard Business Review have an interesting article about what they consider to be Breakthrough Ideas for 2007. There's some interesting ideas in the list, but I find 2 of them more poignant than others:

An emerging hotbed of user-centered innovation

This is one of the most prevalent trends I'm seeing in industry today. Customers aren't happy to sit back and choose from whatever is available on the shelf of their local. They're happier to define something that meets their needs, and get it made, sourced or otherwise procured by watever means necessary.

Take Arthur's birthday party for example. My wife wasn't happy to just buy some standard "Happy Birthday" decorations. Instead, she found a service online that would print a giant banner with whatever text she wanted.

The article sums it up in one apt sentence:

Today, customers aren’t just voicing their needs to companies that are willing to listen; they’re inventing and often building what they want.

Living with Continuous Partial Attention

The other trend that the HBR article identifies is the increased "coping mechanism" that results from dealing with too much information. We are becoming much more selective in what we percieve and we are becoming more skillfull at giving partial attention to a number of information streams simultaneously. HBR predicts that in 2007 business will have to learn to deal with this both in their management style and in their marketing messages.

Should be an interesting year! Hopefully I'll do another catchup on my blog towards the end of the year and see if any of these trends have moved on.

Inside Myspace

2007-01-25T16:12:01Z

Here's an excellent article called Inside Myspace by David F. Carr. It traces the steps MySpace had to go through as they grew from a small website to a social network with technology that needs to handle almost 40 billion page requests a month. The article talks about the architectural decisions that were taken as the scaling requirements for the application grew and grew and grew.

The initial configuration for the website consisted of two web servers talking to a single database server. Initially this was scaled by adding more web servers, but eventually this maxed out the database. The redesign of the system spread the database load across 3 SQL Servers, a master database with two replicated instances. As the web servers were scaled out, the SQL end was scaled out also, but eventually hit a limit where the I/O on the box was just reaching a physcal limit and the site was also exhibiting problems based on the time it to replicate information to all the children.

This spawned another application redesign, this time building around a concept of vertical partitioning, using different databases for different parts of the site. The performance improvements of this redesign were also compounded by moving the databases onto a SAN, which is more performant that physically attached storage. This worked for some time, but as the performance requirements increased, the coupling between the databases started to become the bottleneck. This prompted another redesign, this time moving to a distributed computing architecture.

Another change aspect that hit the project at this stage was the move from ColdFusion to .Net. This brought about performance improvements, not only because .Net is more efficient than ColdFusion, but also because a rewrite forces developers to rethink their code and design for efficiency. By this time MySpace had over 10 million accounts and was starting to max out the SAN's I/O capacity prompting a move to a virtualised storage architecture. The next improvement was a caching tier between the presentation layer and the database. In retrospect, the team acknowledged that this was someting they should have done sooner. The final improvement was to most to SQL Server 2005 which brought about the benefit of a 64-bit architecture and the ability to address more RAM. By 2006, their standard SQL Server configuration consisted of boxes with 64GB of RAM, which brought about much better performance.

It's an excellent article and makes for great reading. If you're interested in system architecture, check it out

ASP.Net AJAX 1.0 ships

2007-01-24T11:33:04Z

I just found out from Scott Guthrie's blog that ASP.Net AJAX 1.0 has just shipped. There's been talk about Atlas and CTP versions of this thing floating around for some time now, but it's nice to see the final version shipping and so much industry support. The whole AJAX initiative makes things much more responsive for end-users and frees developers from browser-specific issues leaving them to focus on real functionality.

Now that Atlas is shipping, it's scheduled to be part of Orcas (which you can already download as a CTP) which will support client-side JavaScript intellisense, JavaScript compilation checking, and rich JavaScript debugging support for ASP.NET AJAX scenarios. Read more about it on ScottGu's blog.

iPhone is finally here

2007-01-10T15:05:29Z

So, the iPhone has hit the market and looks like the gadget to have. There has been lots of talk as to what it could look like (check out the iPhone Concept Blog) but we finally get to feast our eyes on the real thing!

So, what does it consist of? Well, it's:

A widescreen iPod
A touchscreen mobile phone
A wifi-enabled Internet device

And those are just the headline points! It sports some great technology including motion, proximity and ambient light sensors, a multi-touch touchscreen, quad-band GSM, WiFi, EDGE and Bluetooth, visual voicemail and a whole bunch of other really really cool stuff. I NEED ONE OF THESE!!!

Interesting to note that Apple's value increased by a whopping $6.1 billion as soon as the device was announced, while the value of Palm and RIMM dropped $2.2 billion. More here

The Venice Project Beta-Test - Take 2

2007-01-09T09:56:07Z

So, last night I tried to the The Venice Project Beta to work, but ran into a Vista-related issue (software conflict). It was stuck in my mind, so this morning I woke up early to figure out what the problem was before I went to work.

So, first of all, what is the AppPatch folder that Vista uses. Well, it's a folder where Windows stores a database of known DLL issues and their fixes. Here's the official word on how Vista deals with Application Compatability:

Windows XP, Windows Vista, and Windows Server 2003 resolve application compatibility problems by dynamically matching application problems with solutions. The matching mechanism is controlled by a dynamic-link library (DLL) that starts and runs the matching mechanism whenever a user installs or runs an application. However, the matching mechanism is transparent and does not significantly affect the performance of the operating system or installed applications.
The matching mechanism relies on several database files to match problems with solutions. The database files contain a list of applications that have known problems and instructions for resolving those problems. The database files are saved in the \systemroot\AppPatch folder in Windows XP, Windows Vista, and Windows Server 2003.

Okay, useful but I was no closer to getting the package running. Digging through Google a bit more I found 2 ways to deal with the problem:

The better solution:

Find the C:\Windows\AppPatch\AcGenral.dll file.
You need to have Administrator permission, AND do this first to claim your right back in Vista to modify the file...
Right click on file Properties>Security>Advanced>Owner>Edit..>Other users..>Advanced..>Find Now>
YOUR_ADMINISTOT_USER_ACCOUNT>OK> ok to all boxes.
Now rename this files to something as "1AcGenral.dll".

The quick and dirty solution:

Rename \windows\appPatch to appPatchOLD

I tried the quick and dirty solution (knowing full well that this might affect Windows' ability to automatically resolve application compatibility issues), but I wanted to see if I could get Beta up and running .. and

Voila' The Venice Project Beta is now running! Have to run to work now, but I'll give it a spin tonight and report on whether it's any good!

Better Marketing through MySpace

2007-01-09T08:07:38Z

Excellent post on MarketingProfs talking about the benefits, and also the pitfalls of marketing yourself or your company on MySpace. Stephan Spencer talks about some notable failures like Blockbuster UK, 7Eleven, and Meijer as well as some outstanding successes, such as the likes of Apple Computer, the Orlando Magic (the movie studio that produced Superman Returns and Borat), and the musical artist “Weird Al” Yankovic.

If you’re responsible for managing an online presence, whether it’s a personal one or a corporate one; there’s some useful information in the article. The author has been tracking MySpace for a number of months and has noticed some valuable trends. He also shares some insight from a company that claims it attained a measurable degree of success from MySpace, the online jewelry retailer Pugster.

Here’s an interesting statistic I picked up from the article. Did you know that over half the people who visit MySpace are 35 or older; and over two thirds are over 25? So, if you thought it was just a playground for teenagers to mess around and make friends; thing again. MySpace is a powerful force heading your way.

The Venice Project Beta-Test - Vista woes

2007-01-08T22:43:23Z

Just checking my email and noticed I've been approved for The Venice Project's Beta-Testing Program. It's pretty exciting stuff. If you haven't heard about The Venice Project, it's a project being developed by the team behind Skype aiming to change the way TV is transmitted over the Internet. It uses peer-to-peer technology to ensure great quality for everyone watching TV. I had signed up for the beta program some time back, but I've only just got the invitation.

So, I popped down to the Beta Test Website, logged in, downloaded the software, installed it .. and got the following popup:

Looks like the software still isn't quite tailored for Vista, and I still don't know enough about Vista to hack around. Will have to try it out on my XP box.

I don't have any invites yet, but if I get any, is anyone interested in an invite ?

Some cool links

2006-11-28T20:49:34Z

Here's some cool stuff I came across this evening:

XML NotePad 2007 - new version of XML Notepad for browsing and editing XML docs
Fiddler 2 - new version of Fiddler that supports HTTPS
Netmon 3.0 - new version of Microsoft's Network Monitor
NStudy - track your progress as you work your way through the .Net APIs

and my favourite

LCARS controls for .Net - Enjoy !!

Have a good browse ...

SubSonic

2006-11-24T16:19:17Z

Went to lunch with some friends today and Justin was totally raving about SubSonic, a DAL (Data Access Layer) generator that includes Intellisense Integration at design time, scaffolding (Rails-style) and full generation of a project specific DAL (using partial classes so you can regenerate without upsetting your own code). Here's the blurb on the website:

SubSonic is a toolset that helps a website build itself. At it's core it's:

A Data Access Layer (DAL) builder that requires no code on your part, it builds itself at compile-time with a full object layer and strongly-typed collections
A complete utility toolset, complete with Rails-like scaffolding, migrations (DB Versioning), and code generators
A dynamic query tool, that lets you use SQL Server and the Enterprise Library without having to know SQL
An OR Mapper that extends to views and stored procedures so you're not locked into the OR/M thing

Might look at it one of these days. In the meantime, check it out here. There's also a demo webcast here

BizTalk 2006 SAP Adapters

2006-11-20T22:55:35Z

There's a couple of SAP Line of Business Adapter Walkthroughs on the BizTalk 2006 Adapter's Page on MSDN, which might be interesting if you planning to work with SAP. They are:

Check them out. The second one is 25 pages long, so don't expect it to be a simple point and click (as most integration marketing demos would have you believe .. but don't get me started on that one ....)

Business Oriented Architecture

2006-11-16T21:50:30Z

Interesting article on bpm.com about Business-Oriented Architecture by Rod Favaron. In it he talks about how business-orientation is what makes BPM different from services-oriented architecture (SOA). For a business person, the chief value of a SOA is that it removes the need to understand the complex application infrastructure of the company today; this simplifies the conversation – and returns the discussion of process improvement to a business orientation. BPM also provides a set of tools for business users; tools focused on business improvement, not service implementation and management. The ability to quickly and flexibly define end-to-end processes, identify performance goals and then manage to those goals is what sets BPM apart from technology-based tools. Read more about it here