Friday, February 17, 2006
Software Development's Evolution towards Product Design
Then comes the inevitable question, “Oh, so you are a programmer?” A gleam appears in my eye and I no longer feel obligated to blather on about the rainy weather. With a great flourish, I whip out my gold nibbed pen and draw a little diagram on a napkin that explains concisely how modern software development works. In the grand finale, I circle one of the little scribbles buried deep in the entire convoluted process and proudly proclaim ‘And that is what I do!”. This admittedly selfish exercise usually keeps everyone involved merrily entertained until dessert arrives.
Download the printable PDF version
After dozens of napkin defiling lectures, I’ve put together an extended PDF of my sketch for download. In short, we have a one page infographic that explains:
- The evolution of software development over four distinct eras.
- The key goals of software development and our saddest failures
- Where software development is moving in the future.
You can download the full PDF here. Print it, share it with your friends. Read on to hear how we got to this point.
The Technocrat Era: Programmers serving programmers

At the dawn of software history, programmers wrote software for other programmers. This was a golden era. Life was so simple. The programmers understood their own technical needs quite intimately and were able to produce software that served those needs. The act of software development was a closed circuit. A programmer could sit in a corner and write code that he wanted. By default it also happened to apply to other programmers.
Programmers that grew up in these idyllic days still remember it fondly. There is still the programmer who will claim that all they need is EMACS and the latest version of GCC to make great software. For software intended for a highly technical audience, they may well be right.
The Early Business Era: Programmers attempt to serve others

The end of the Technocrat era was came about due to a startling discovery: the vast majority of the world was composed of non-technical people. Artists, accountants, authors, history majors and other unexpected consumers roamed the prehistoric landscape outside the hallowed engineering halls.
A new class of software entrepreneur called a ‘business man’ came into existence. This new creature realized that the hordes of non-technical people had their own needs that might be served by a well written program running on one of these new fangled “personal computers.” The business man gathered programmers and told them to build software to solve things like balancing budgets or writing letters.
The software products that the programmers created were technical marvels. They provided practical benefits far beyond what was available. VisiCalc revolutionized finance. WordStar and WordPerfect forever changed the act of writing. Even the Quantel Paintbox changed the world of art.
And yet the recently converted users, who couldn’t live without the efficient, powerful and technically amazing new software, were curiously ambivalent. They liked what these new products did from a practical standpoint, but found them to be confusing and often quite irritating.
You see, the programmers treated their customers just like programmers. They made them memorize crazy expert keyboard conventions and loaded the product with dozens of obscure features. This is what the programmers wanted out of a piece of software, so they assumed that the customers must want the same.
Unfortunately, the customer needs were a different beast. Their needs could be roughly divided into two categories:
- Practical needs: They wanted a product that worked. For example, the product obviously needed to save time and money.
- Emotional needs: They also wanted products the possessed less tangible benefits. They wanted applications that treated them with kindness and understanding if they made a mistake. They even wanted to use products that were attractive and conferred status. The customers desired programs that appealed to the softer aspects of their humanity.
The Late Business Era: Programmers and artists meet and do battle

The software market had become quite competitive at this point. Business folks, experienced in the Jedi wisdom of more mature markets, reasoned that perhaps serving emotional needs could give their companies an edge. A few companies experimentally hired non-programmers such as artists, marketing and usability dabblers. I use the term artist quite liberally here since it captures that charming, hand-waving vagueness of all classes of “people people”
Oh, the suffering that resulted. The inevitable culture clash was a bit like unleashing wild dogs upon one another and then watching them sulk afterwards.
None of the freshly introduced team members spoke one another’s language. The artists talked about fluff like color and mood. The marketing people made outrageous requests with zero comprehension about technical feasibility. The programmers were suddenly enslaved by bizarre, conflicting feature demands that they did not understand. “Make it friendlier” translates poorly into C++ code.
Let’s take something as simple as making an interface more appealing.
- The artist would whip up a picture sporting rounded corners and more pleasing colors. They’d send it to over to the programmer with a hand scribbled note “Make this.”
- The programmer then would either A) State that an infinite number of programmers could never finish such a technical abomination or B) Recreate the image using rectangles and the preexisting color scheme.
- Everyone would then rage at one another about their general incompetence.
The best products came from those odd teams that managed to compromise. The technology was clumsy and the emotional benefits of the software shaky. But it was better than the crap that customer had to put up with before. The original Mac OS was a great example of this. Later versions of Windows also managed to address a few emotional needs.
The Product Design Era: Can programmers and artists learn to work together?

The clock of progress has moved forward once again. The competition ratcheted up one more notch, and it appeared evident that companies that fail to master the lessons of the last era would be punished by the customers. Our battle scarred software companies were left looking for a better way.
Obviously, designing for emotional needs in addition to technical needs was a winning evolutionary strategy. In the last era however, mixing multiple potent skill sets together left organizational and cultural wounds that often were difficult to heal.
The companies that survived the influx of designers and marketing folks often relegated the survivors to their own separate silos. Marketing people got one org tree, developers got another. Artists and usability folks in most cases were shoved in random back corners to rot in black despair. Many groups were so busy protecting their domain that it was surprising that software got released at all. Release dates slipped by multiple years as the development talent stagnated is a cesspool of misplaced process.
What was needed was the most dramatic of transformations: A change in the cored development process. What made it difficult is that original culture of technocratic software development would need to evolve to support a broadly humanistic approach to product design.
The lead users of the Product Design Era
This change found root, as is the case for most dramatic transitions, from the most unlikely places.
There is a concept in product design known as the ‘lead user’. This is a group of users that solves a difficult program far ahead of the mass market. Often they’ll cobble together their own tools and discover fundamental process and technology issues years in advance of mainstream users.
The old joke about writing Shakespeare with infinite monkeys randomly typing on typewriters got it all wrong. Instead give me a million customers trying to do their job with broken tools and one of them will stumble upon a process that is truly better. By watching the edges of the market place, we gain great insight into the direction that the larger market will take in the future.
The lead users in software development came from several widely divergent areas.
- The first was the game industry. Here, small cross functional teams built products focused entirely on serving emotional needs. Due to intense competition and vicious delivery cycles, many teams were forced to innovate far outside the traditional software development methodologies.
- The second were companies like Apple that followed curious ‘design’ philosophies more similar to that pursued by consumer good companies than software companies. What do shampoo companies and software development have in common? A remarkable amount it turns out.
- The third were web design companies. Due to the low cost of entry and the early emphasis on the web as a marketing medium, the web design market is dominated by tiny, hungry graphic design firms. They brought with them a culture of small teams, close collaboration between artists and programmers, and a nearly slavish devotion to serving their customers.
Each of these lead users follows a variant of what is broadly known in more mature industries as “product design.” They see software development not as a pure technical exercise. Instead they look at it as an integrative process of building a new product that solves both their customer’s combined emotional and practical needs. Even when forced to use a “technically inferior” platform, the religious devotion to rapidly and effectively serving customer’s complete spectrum of needs make their product offerings more attractive than the competition.
Robert G. Cooper, a well known researcher on new product development, states that there are several core factors (listed in order of importance) for any successful new product design process:
- A unique, superior and differentiated product with good value-for-money for the customer.
- A strong market orientation – voice of the customer is built in
- Sharp, early, fact-based product definition before product development begins
- Solid up-front homework – doing front end activities like market analysis well
- True cross functional teams: empowered, resourced, accountable, dedicated leader
- Leverage – Where the project builds on business’s technology and marketing competencies
- Market attractiveness – size, growth, margins
- Quality of the launch effort: well planned, properly resourced
- Technological competencies and quality of execution of technology activities.
Software companies that understand product design tend to pour their efforts into the following activities:
- Focus on a unifying team goal built around customer needs. They ensure that the product is always driven by customer needs, not internal whims.
- Work together in cross functional teams. They build an organization that encourages the sharing of skills to promote problem solving. They discourage the formation of silos of individual ‘experts’.
- Communicate clearly with a process that applies to all team members. They build a commonly followed process includes both fuzzy front end activities like design and production activities like coding. This process forms the language that unifies disparate groups.
- Work efficiently with design friendly tools. The team avoids custom coding and ‘tossing it over the fence’ by adopting tools that work on common data format across all skill sets.
The benefits of a product design process are well documented. New products that deliver superior, unique benefits to the customer have a commercial success rate of 98% compared to 18.4% for undifferentiated products. These products reach an outstanding 53.5% market share.
Some of the highlights of a strong product design include:
- Create highly competitive products that achieve market dominance.
- Save money by focusing on the right features that bring customer value, not low yield ‘nice-to-have’ features.
- Create passionate customers that accelerate the spread of your marketing message.
The answer will be simple. The successful company identified the correct emotional and practical needs of the customer and poured their efforts into serving those needs. The richer companies - flush with silos of ‘wise’ experts - fought with one another and threw random features at the customer. It is rarely about doing more; it is about grokking customers and doing the small set of correct things necessary to succeed.
Dangers of the Product Design Era
As with any process transition, some groups are adopting these techniques slower than others.
- Even within game development there are huge swaths of publishers and development teams that are ignorant of techniques used to incorporate market research, concept testing and new-to-the world innovation into their process.
- Many web developers still create new products by following their ‘gut’ without clearly identifying their ultimate customer.
- Most traditional desktop software developers are just barely escaping the Late Business era of functional silos and warring factions.
Unfortunately, many companies that attempt to adopt a product design philosophy will also fail, despite their best efforts. Cultural change is hard work. To adopt product design you must alter the most basic DNA of the company’s values.
- It involves asking vice presidents to give up their empires that they’ve fought decades to establish. What is the point of an organization of engineers when engineers are all just members of small cross-functional teams?
- It involves asking the men and women in the trenches to give up their own dreams of building their own empires according to the old rules.
- If the ultimate reward of the old system is an isolated corner office with windows, try convincing people to work together in a common war room that has walls covered with whiteboards.
Such traumatic change is absolutely necessary. If you want to make great products for happy customers, you need to make the transition to a broader product design methodology. We may have once been a clique of technocrats. But now we must take our place in the broader society by providing human solutions to the very human customers that we serve.
Dessert
After I’m done drawing all this out on my little napkin, I point to one of the designer fellows in a beret and say, “See that guy? That’s me. I work on a team full of crazy, wildly talented people trying to make the world a better place. We make products that you can love, not just products that you use.”
Folks nod. They get it. They like the idea of software that not only works, but makes them feel good about using it. At one meal I was sitting with an older woman who listened most politely to my wildly gesticulating explanation of modern software development.
At the end she said, “Well, it is about time.”
Take care
Danc.
PS: This cute little illustration was done in a program I'm busy renovating called Expression Graphic Designer. It will be part of a suite of design friendly tools that seeks to ease the bloodshed between programmers and their more artistic team members. You can't build a cross functional production pipeline without the right tools.
References and Random Clippings
- Success factors in new product development
Winning at New Products, pg 58-59, Robert G. Cooper - What we can learn from the Stratocaster http://www.cooper.com/newsletters/2001_07/what_we_can_learn_from_the_stratocaster.htm
- Inmates running the asylum (a book review)
http://www.wap.org/journal/inmates.html - The Apple design team work space
“We have assembled a heavenly design team. By keeping the core team small and investing significantly in tools and process we can work with a level of collaboration that seems particularly rare. Our physical environment reflects and enables that collaborative approach. The large open studio and massive sound system support a number of communal design areas. We have little exclusively personal space. In fact, the memory of how we work will endure beyond the products of our work.” – Jonathan Ives
http://www.designmuseum.org/design/index.php?id=63 - Lead user analysis
http://www.betterproductdesign.net/tools/user/leaduser.htm
http://www.leaduser.com/documents/lu_perform_study_wp_enthusiast.pdf - Concept of total customer experience
http://www.findarticles.com/p/articles/mi_qa4001/is_200201/ai_n9056014 - When worlds collide: Integrated development with business and design students
http://www.findarticles.com/p/articles/mi_qa4001/is_200207/ai_n9103531 - When innovators stop innovating
http://www.findarticles.com/p/articles/mi_qa4001/is_200204/ai_n9024018 - Rounded Rects at Apple: Benefits of cross functional discussion
http://www.folklore.org/StoryView.py?project=Macintosh&story=Round_Rects_Are_Everywhere.txt - A tech writer and a programmer prove the benefits of cross functional teams
http://www.folklore.org/StoryView.py?project=Macintosh&story=Inside_Macintosh.txt - Development of Mac Paint
http://www.folklore.org/StoryView.py?project=Macintosh&story=MacPaint_Evolution.txt&sortOrder=Sort%20by%20Rating - Use of cross functional teams to improve game development:
http://therubrabbits.1up.com/do/blogEntry?bId=6530632&publicUserId=5729773#comments
'Sleeping on it' best for complex decisions
Complex decisions are best left to your unconscious mind to work out, according to a new study, and over-thinking a problem could lead to expensive mistakes.
The research suggests the conscious mind should be trusted only with simple decisions, such as selecting a brand of oven glove. Sleeping on a big decision, such as buying a car or house, is more likely to produce a result people remain happy with than consciously weighing up the pros and cons of the problem, the researchers say.
Thinking hard about a complex decision that rests on multiple factors appears to bamboozle the conscious mind so that people only consider a subset of information, which they weight inappropriately, resulting in an unsatisfactory choice. In contrast, the unconscious mind appears able to ponder over all the information and produce a decision that most people remain satisfied with
Ap Dijksterhuis at the University of Amsterdam in the Netherlands, and colleagues recruited 80 people for a series of lab-based and “real-world” tests. The participants were provided with information and asked to make decisions about simple and complex purchases, ranging from shampoos to furniture to cars.
Snap decisions
In one of the tests, half of the participants were asked to ponder on the information they were given and then decide which among similar products to buy. The other half were shown the information but then made to perform a series of puzzles including anagrams and simple arithmetic. At the end of the puzzle session, the participants were asked to make a snap decision about the products.
“We found that when the choice was for something simple, such as purchasing oven gloves or shampoo, people made better decisions – ones that they remained happy with – if they consciously deliberated over the information,” says Dijksterhuis.
“But once the decision was more complex such as for a house, too much thinking about it led people to make the wrong choice. Whereas, if their conscious mind was fully occupied on solving puzzles, their unconscious could freely consider all the information and they reached better decisions.”
Expectation counts
However, the unconscious mind appears to need some instruction. “It was only when people were told before the puzzles that they would need to reach a decision that they were able to come up with the right one,” Dijksterhuis told New Scientist.
If they were told that none of what they had been shown was important before being given the puzzles, they failed to make satisfactory choices.
“At some point in our evolution, we started to make decisions consciously, and we’re not very good at it. We should learn to let our unconscious handle the complicated things,” Dijksterhuis says.
source:http://www.newscientist.com/article.ns?id=dn8732&feedId=online-news_rss20
Alzheimer's Progresses Faster in Educated People (Update1)
Feb. 16 (Bloomberg) -- High levels of education speed up the progression of Alzheimer's disease, according to a study published in next month's issue of the Journal of Neurology, Neurosurgery and Psychiatry.
Mental agility dropped every year among Alzheimer's disease patients with each additional year of education, leading to an additional 0.3 percent deterioration, the researchers from the Columbia University Medical Center in New York found. The speed of thought processes and memory were particularly affected.
Previous studies have shown that people with high levels of education are less likely to develop Alzheimer's disease. The new study shows that the brains of more educated people can tolerate changes for longer periods of time, meaning signs of decreased mental agility typical of Alzheimer's disease appear later. When those signs do appear, the disease progresses faster than it does in less educated patients.
``The amount of nerve connections and information hubs are likely to be more numerous and more efficient in people who are highly educated,'' said lead author Nikolaos Scarmeas in his study. ``The subsequent impact is likely to be greater than it would be in less educated brains, because of the higher levels of accumulated damage.''
The findings are based on the study of 312 New Yorkers aged 65 and older, who were diagnosed with Alzheimer's disease and monitored for more than five years. All the patients underwent around four neurological assessments, each of which comprised a dozen separate tests of brain function.
The level of the drop-off was particularly evident in the speed of thought processes and memory, Scarmeas said. The result didn't depend on age, mental ability at diagnosis or other factors likely to affect brain function, including depression and diseases of the blood vessels.
The number of people who have dementia probably will double every 20 years to 42 million by 2020 with one new case every seven seconds, according to a report about Alzheimer's disease published in the Lancet in December. Alzheimer's disease is the most common cause of dementia in people 65 and older.
source:http://www.bloomberg.com/apps/news?pid=10000102&sid=alzuc18R8oGc&refer=uk
Microsoft hopes prizes attract new search users
Remember the long-running e-mail hoax from the early days of the Web, the one that had Bill Gates testing an "e-mail tracing program" and offering to pay recipients big bucks if they passed his test e-mail along to all their friends?
Well, the offer is true, sort of. Microsoft Corp. wants you to use its MSN search engine, and it's got $1 million worth of prizes up for grabs for those who nibble at the offer.
The company announced the three-month promotion Tuesday, inviting Web searchers to visit MSNSearchandWin.com to take their chances at winning a prize.
source:http://www.msnbc.msn.com/id/11355791
How Do You Store Your Previously-Written Code?
source:http://ask.slashdot.org/article.pl?sid=06/02/16/2344228
Mazda Plans Dual-Fuel Car in Japan
TOKYO — Japan's Mazda Motor Corp. said on Wednesday it will begin leasing a dual-fuel car that can run on both hydrogen and gasoline in the auto industry's latest effort to reduce oil consumption in vehicles.
Most major auto makers are developing zero-emission hydrogen-powered fuel cell cars as a potential alternative to today's conventional gasoline and diesel engine cars but believe they are decades away from mass production due to high development costs and lack of infrastructure.
Mazda said the RX-8 Hydrogen RE, based on its popular RX-8 sports car, gets around these problems by running on gasoline in the absence of a hydrogen fuelling station, and using existing engine parts and production facilities to lower costs.
The car is powered by Mazda's iconic rotary engine and can switch between hydrogen and gasoline fuel with the flick of a switch. It can cruise for a maximum 62 miles on hydrogen and 549 km (341 miles) on gasoline, it said.
Fuel cell cars, meanwhile, use hydrogen to first generate electricity through a fuel cell stack for power, and require an electric motor.
A rotary engine is suitable for hydrogen fuel because the separate chambers for fuel intake, combustion and exhaust significantly reduce the danger of the fuel's backfiring compared with a conventional recipro engine.
Internet Governance: An Antispam Perspective
All those Internet Governance pundits who track ICANN the way paparazzi track Paris Hilton are barking up the wrong tree. They’ve mistaken the Department of Street Signs for the whole of the state. The real action involves words like rbldnsd, content filtering, and webs of trust.
Welcome to the Internet! What’s on the menu today? Spam, with some phish on the side! We’ve got email spam, Usenet spam, IRC spam, IM spam, Jabber spam, Web spam, blogs spam, and spam splogs. And next week we’ll have some brand new VoIP spam for you.
Now that we’re a few years into the Cambrian explosion of messaging protocols, I’d like to present a few observations around a theme and offer some suggestions.
Just so you know where I’m coming from, the foremost concern in my mind is this: The final solution to the phishing problem requires that people use a whitelist-only, default-deny paradigm for email. Many people already subscribe to default-deny for IM and VoIP, but there is a cultural resistance to whitelist-only email—email is perceived as the medium of least reserve. I believe that we must move to a default-deny model for email to solve phishing; at the same time we must preserve the openness that made email the killer app in the first place. The tension between these poles creates a tremendous opportunity for innovation and social good if we get things right, and for shattering failure if we get things wrong. Can you imagine a Balkanization of messaging, where if you want to talk to someone you have to first join their BBS? I’m an idealist: I care deeply about the future of free communications. I don’t want to screw this one up.
The following points help me develop my argument.
1. Every open medium can be abused.
2. Emigrating from a hostile environment and jailing violators are two sides of the same coin.
3. Extradition only works when governments agree.
4. National borders don’t work online; we need new kinds of boundaries.
5. DNSBLs are the prisons of Internet email.
6. Let’s create a world where the consensus reality is as inclusive as possible.
One: Every open medium can be abused.
The abuse comes in a hundred different forms, but deep down they’re all, somewhat tediously, the same: bad guys telling lies for profit. (Do you remember the one exception, the kook looking for a time machine?)
Why do closed media suffer less abuse? Because closed media have centralized architectures, built-in authentication, real-world identities, prerequisites for access, well-defined usage guidelines, and short paper trails. Of course they’re better at kicking offenders off the network. They have staff paid to do just that!
Open systems, by design, are more of a free-for-all.
So why don’t we just build one big closed system? Many youngsters have given up on email, and prefer to communicate over Myspace and AIM. If you want to talk to them, you have to sign up with Myspace and AIM. If you cause trouble, Myspace and AOL reserve the right to kick you off the system. The economic costs of signing up—the time it takes to sign up, pass CAPTCHA, and learn how to use the system—are analogous to other economic costs that have been proposed, such as “penny per email”. And enforcement can be much more effective.
Just as the free market has voluntarily chosen a monopoly regime for desktop operating systems, maybe the free market will eventually choose a monopoly regime for messaging systems. We may simply find that it’s cheaper to pay one vendor to manage spam for everyone.
But I doubt it; just as the Microsoft mainstream fuels the Linux and OS X countercultures, any messaging mainstream will fuel alternative modes of communication. Why?
Because I might not want to live in your country. If World of Warcraft—a very complex messaging system—declares itself a gay-free zone, where will gay gamers (gaymers?) go? They’ll go elsewhere. Time and again we have seen governments declare a uniform standard for behaviour: time and again we have seen people get up and walk away.
Now we’re getting into politics and governance. You might want to go make a cup of tea: this argument is about to detour down the scenic route.
Two: Emigrating from a hostile environment and jailing violators are two sides of the same coin.
Much of mankind’s political history can be traced to the idea of voting with your feet. If you don’t like the way things are done in one place, you go someplace else.
When the Mennonites left Europe for America in the 18th Century, they were simply implementing “no thanks, I’m leaving”.
Emigrating away from a hostile domain is a bottom-up approach. Symmetrically, this pattern has a top-down version. Mennonites enforce community standards by shunning. In every society, people who don’t play by the rules get sent to prison or exiled. And that’s society implementing “no thanks, you’re leaving.”
(Populating Australia with criminals implements both approaches at once.)
At the G2G level, when countries get mad at one another, the first thing they do is break off diplomatic relations and pretend the other country doesn’t exist.
But that only works to a limited degree. One man’s free speech is another man’s blasphemy. And, as certain Danish cartoonists and European newspapers have recently discovered, globalization makes it real hard to ignore the sins of your neighbours. Some societies are not content to apply their values locally—“we don’t keep dogs as pets, but we don’t mind if you do”—but wish to apply them globally—“we don’t draw certain kinds of pictures, and nobody else may either.”
But online, the feet are virtual, and every place is, in some sense, everyplace else. This means that, online, we need to come up with new ways for people of one mind to “migrate” away from people of another.
Three: Extradition only works when governments agree.
Most developed countries in the West generally want to help catch one another’s criminals. But, as we noted above, community standards differ. Governments get particularly touchy in matters of jurisdiction and sovereignty. So Roman Polanski lives in France, unmolested by the United States.
If you sue spammers in Tampa, they pop up again in Taiwan. Extraditing skript kiddies just doesn’t scale. National borders don’t work online.
That’s what makes it so hard to police the Internet. The very idea of policing goes hand in hand with the idea of jurisdiction. And jurisdiction ultimately goes back to the idea of a state.
Nation-states have been around in their modern, governmental form, for a couple hundred years now. The modern Internet has been around for maybe twenty. If nation-states are a horse and buggy, the Internet is a hybrid automobile.
Asking a nation-state to manage Internet crime is like asking a Mountie to pull over a Prius.
Four: Our global village has no borders.
Ten years ago, back when we all thought global villages and information superhighways were just the bee’s knees, hordes of breathless futurists proclaimed the Internet has no borders! Borderless was good, hot, fun!
Now we’re discovering that organized crime loves the Internet precisely because it has no borders, no jurisdiction, no police.
Well, almost no police. Ten years after we built the information superhighway, you can’t spend ten minutes on 101 without seeing a Cisco billboard announcing “trojan horse corralled” or “denial of service denied”. (The whole ad campaign feels bafflingly insider, sort of like a postmodern Burma Shave, but authored by mildly autistic types who simply don’t care whether anybody else understands what they’re saying.) But what does it mean? It means Cisco is beginning to provide infrastructure on the Internet the way Halliburton provides infrastructure in Iraq: they build the roads and they man the checkpoints.
Do we really want a centralized authority integrated into the infrastructure, or do we want the ability to choose which communities we want to live in? We can take power back into our hands and draw our own borders…
Five: DNSBLs are the prisons of the Internet.
Spamfighters like to think of DNS blacklists as a cutting-edge tool for the 21st century, but I would wager that if you went down to Lancaster, PA, flagged down an Amish farmer driving his 18th century horse and buggy, and showed him a DNSBL, he would recognize it instantly: “you’re shunning those who can’t hold their tongues!”
Just as shunning is a frightfully effective form of Amish social control, DNS blacklists are a frightfully effective form of Internet social control.
The interesting thing about blacklists is that you get to choose which ones you want. Today this just means that if you pick a DNSBL that’s a little bit too activist, you get some false positives, some intentional collateral damage. But tomorrow, if we flip the switch from default-accept to default-deny, picking the wrong whitelists, or too few of them, could result in eclipsing whole swaths of the Net.
That would be bad. It would also be dangerously tempting. Most people don’t know what they don’t know. It’s very easy to whitelist all the good people in your addressbook, but it’s harder to whitelist all the good people not in your addressbook. If we want to keep email as open as it used to be, we have to be very inclusive. This is where reputation systems come in: just as credit bureaus tell financial institutions if someone is likely not to pay their bills, reputation systems tell mail receivers if someone is likely to be a spammer. Reputation systems are essential to solving the first- contact problem, but that is a topic for a different article.
Crudely but functionally, a consensus reality begins with a group of persons who recognize each other’s existence. When we use DNSBLs, we refuse to communicate with the entities listed—we killfile them, but the killfiles are shared. I’m told that Spamhaus covers something like half a billion mailboxes: that’s half a billion mailboxes who don’t talk to anyone listed on the SBL.
In a default-deny world, we’re less interested in the bad guys and more interested in the good guys. I could easily imagine a world where you whitelist everyone in your addressbook, everyone in your family, your company, your industry network, your church, your school, your neighbourhood ... and that might get you good enough coverage that you wouldn’t notice the legitimate messages that are getting blocked.
Six: Let’s create a world where the consensus reality is as inclusive as possible.
What have we learned?
We counter abuse in open media on the borderless Internet not by bringing antiquated legal instruments to bear, but by collectively agreeing on whom to shun and whom to put on the guest list.
Shunning is what you do when your default rule is “allow”—when you’re liberal in what you receive, you expect others to be conservative in what they send. When they’re not, you killfile them.
In open media, though, we’re discovering that maybe the default rule should be “deny”. In big cities, we’ve learned, tediously, that most strangers who approach us on the street are trying to tell us lies for profit. The Internet has become a big city. When others are liberal in what they send, maybe we should be conservative in what we receive. The opposite of a blacklist is a whitelist. The opposite of shunning is selective introductions—think Victorian social norms. Whole industries are springing up around these ideas: accreditation and reputation are hot topics in the antispam world today, but the ideas behind them are as old as human civilization. If there is nothing new under the sun, then let us learn from history and not repeat it!
We need mechanisms to agree on who’s worth talking to and who’s not. First-generation, proprietary, integrated approaches like Goodmail will pave the way for next-generation technologies based on open standards. And these technologies will lay the foundations for what I consider to be the meat of Internet governance: rules that determine precisely to what extent my Internet overlaps with your Internet, and whether I let you into my world at all. In short, whether the walled garden I build for myself opens on the walled garden you build for yourself.
After all, if there are no national boundaries on the Internet, then maybe, just maybe, every man is an island, entire of itself. Each of us builds bridges to our neighbours, and we roam on those bridges until we feel uncomfortably far from home.
As a matter of public policy, I want to set a goal: in a world of default deny, let’s use all the tools at our disposal to make our walled gardens as big as they can be. Why? Because, as Donne said, “I am involved in mankind”; if you are not in my whitelist, I may not be in yours, and we are both the poorer for it.
source:http://www.circleid.com/posts/internet_governance_an_antispam_perspective/
Toxic Toads Evolve Long Legs and Take Over Australia
Cane toads (Bufo marinus) were first brought in from Hawaii in 1935 to control the spread of beetles that were ravaging Australia's sugar cane crop. But instead of controlling the pests, the toads have become pests themselves. A deadly chemical defense system disposes quickly of potential predators.
The toads have expanded their range to cover more than a third of Australia's total land area.
Hop to it
From the 1940s through the ‘60s, the toads were invading at a rate of about 6 miles per year; now they're taking over at a rate of about 30 miles a year.
To find out why the toads are spreading so fast, researchers stationed themselves about 40 miles east of Australia's port city of Darwin, in a region where the cane toads had not yet spread.
When the toads arrived, the researchers found that those in the vanguard of the invasion had legs that were up to 6 percent longer than average; shorter-legged stragglers followed. The study showed that newer populations of toads tended to have longer legs than those in long-established populations.
A top pest
It should come as no surprise that cane toads are among the world's top 100 invasive species. They are the world's most introduced amphibian. They mate year-round and females lay up to 30,000 eggs at a time.
The toads can grow as large as dinner plates and weigh up to 4.5 pounds. Their heads and backsides are studded with rows of warts that secrete a milky white toxin called bufotoxin.
Because Australia has no native toads, many native predators such as snakes, lizards and mammals are very sensitive to the toxin. So when the toads spread, they immediately kill off many of the region's top predators.
"We don't know what effect it may have to remove so many top predators from a complex tropical ecosystem, but it's likely to be bad news," said study team member Richard Shine of the University of Sydney.
Ecological disaster
When an invasive species is first introduced, the population remains low for a few generations before exploding, Shine said.
"It's likely that such lags reflect, at least in part, adaptive changes in the invader to suit it to the new environment," Shine told LiveScience.
Shine and his colleagues warn in the February 16 issue of the journal Nature that Australia could face an "ecological nightmare" if the spread isn't controlled soon.
Australian scientists have tried for decades to eradicate the toads, but with limited success. Last year, researchers announced they had successfully lured and trapped the toads using ultraviolet lights like those used in disco clubs.
- Invasive Species: Photo Gallery
- Invasive Creatures Attack Like Internet Viruses
- Wanted, Dead or Alive: Exotic, Predator Frog in Georgia
- Evolution's Useless Limbs and Vestigal Organs
Visit LiveScience.com for more daily news, views and scientific inquiry with an original, provocative point of view. LiveScience reports amazing, real world breakthroughs, made simple and stimulating for people on the go. Check out our collection of Amazing Images, Image Galleries, Interactive Features, Trivia and more. Get cool gadgets at the new LiveScience Store, sign up for our free daily email newsletter and check out our RSS feeds today!
source:http://news.yahoo.com/s/space/20060215/sc_space/toxictoadsevolvelonglegsandtakeoveraustralia
Continued Success for Space Elevator Tests
Seattle, Washington - Feb. 13, 2006 - LiftPort Group, the space elevator companies, today announced that it has successfully completed its second round of preliminary tests of its high altitude platform and robotic lifters. The tests, which were conducted under a waiver to use airspace granted by the Federal Aviation Administration (FAA), used prototypes of proprietary technology the company is developing for use in the LiftPort Space Elevator, the company's revolutionary way to ferry cargo into space.
In this phase of testing, conducted earlier this month in Arizona, LiftPort successfully launched an observation and communication platform a full mile in the air and maintained it in a stationary position for more than six hours while robotic lifters climbed up and down a ribbon attached to the platform. The platform, a proprietary system that the company has named "HALE" (High Altitude Long Endurance), was secured in place by an arrangement of high altitude balloons, which were also used to launch it. The robotic lifters measured five feet, six inches and climbed to a height of more than 1500 feet, surpassing its last test record by more than 500 feet.
"We're pleased at the success of this round of testing," said Michael Laine, president of LiftPort. "Testing our technology in real world settings is critical to the ultimate success of our space elevator, and we appreciate the FAA's willingness to work with us on this."
In addition to the LiftPort Space Elevator, the LiftPort HALE system has other near term commercial applications that the company plans to develop and market. These include security, high altitude observation cameras, acting as a relay station for radio, cellular or Internet access during natural disasters, or for real time surveillance over the damaged region.
A revolutionary way to send cargo into space, the LiftPort Space Elevator will consist of a carbon nanotube composite ribbon eventually stretching some 62,000 miles from earth to space. The LiftPort Space Elevator will be anchored to an offshore sea platform near the equator in the Pacific Ocean, and to a small man-made counterweight in space. Mechanical lifters are expected to move up and down the ribbon, carrying such items as people, satellites and solar power systems into space.
Headquartered in Bremerton, Wash., LiftPort Inc. is a privately held company dedicated to the development of the first commercial elevator to space. For more information, or to sign up for a free subscription to the company's newsletter on the LiftPort Space Elevator, visit at the company's web site at www.liftport.com.
source:http://www.liftport.com/index.php?site=news&news_id=3
Google Acquires Measure Map
source:http://slashdot.org/article.pl?sid=06/02/15/1732240
Big Brain Thinking
Scientists are learning volumes about the brain -- how it can make split-second decisions, how it learns from past mistakes, how it converts pulses of light into a complex visual scene. But, for some, deciphering the "language" of the electrical pulses that travel through our brains is only half the story. The second part, and one that is far more philosophical and complex, is how that brain activity translates into consciousness -- a person's self-awareness and perception of the world around them.
Bill Newsome, a neuroscientist at Stanford University in Palo Alto, CA, has spent the last twenty years studying how neurons encode information and how they use it to make decisions about the world. In the 1990s, he and collaborators were able to change the way a monkey responded to its environment by sending electric jolts to certain parts of its brain. The findings gave neuroscientists enormous insight into the inner workings of the brain.
But Newsome is obsessed with a lingering question: How does consciousness arise from brain function? He feels the best way to answer that question is by implanting an electrode into his own brain -- and seeing how the electric current changes his perception of the world.
Newsome would not be the first person with a brain implant. Epilepsy patients undergo electrical stimulation prior to brain surgery. A paralyzed man in New England has an experimental implant that translates his brain activity into movements of a robotic arm. And, perhaps most famously, Kevin Warwick, a cybernetics professor at the University of Reading, U.K., first implanted a chip into nerve fibers in his arm in 2002, then implanted a chip in his wife's arm, as part of his quest to become a cyborg.
It's not certain that Newsome will get approval for such a radical undertaking. But, if he does, his experiment won't be in the interest of curing a disease or become a human machine. He's hoping to do something broader: understand consciousness.
Technology Review: Why is understanding consciousness so important to you?
Bill Newsome: I think that how consciousness arises out of brain function is one of the most fascinating and important questions in all of neurobiology. If we understand the system completely (from input to output) at a cellular level, but still do not know exactly what causes conscious mental phenomena, we will have failed.
TR: Most of your experiments have been done on monkeys. How did that begin to shape your view on the relationship between brain functions and human consciousness?
BN: We study motion perception. We train monkeys to look at a pattern of dots moving in a certain direction and to report the direction of the dots by moving their eyes in the same direction. If a monkey picks the correct answer, he gets a reward.
This simple behavior contains a world in terms of understanding how the nervous system performs intelligent behavior. Sensory information that comes into the brain through the eye must be coded into some neural language that represents the stimulus within the brain. Based on this neural representation, the monkey must then make a high-level judgment about what he is actually seeing. This "decision" in turn guides the selection of a motor response, to look to the left or the right.
TR: And you added a new level to this experimental setup by stimulating the monkey's brain.
BN: We put an electrode in an area of the brain known as MT. The cells in this area respond selectively to a specific direction of motion. Some cells are active when the monkey looks at dots moving to the left, some cells are active when the monkey looks at dots moving to the right. People had suspected for a long time that MT was important for our ability to see motion. So we did an experiment where we stimulated these cells artificially with tiny pulses of electrical current -- it changed what the monkeys reported seeing.
TR: So with the monkey experiments, you can stimulate the brain in very focused ways and change the way the monkey responds. But the monkey can't tell you what he sees when you stimulate the brain.
BN: Yes. People can report what they see or hear or feel, but with monkeys, you can only look at their change in behavior. I can't climb into a monkey's head and see what the monkey really sees.
This gets to core of the current debate about the study of consciousness. What is the conscious experience that accompanies the stimulation and the monkey's decision? Even if you knew everything about how the neurons encode and transmit information, you may not know what the monkey experiences when we stimulate his MT.
TR: People have shown that stimulating the human brain can do similar things too, right?
BN: Electrical stimulation of the brain is not new. Wilder Penfield, a neurosurgeon in Canada in 1930s and 40s, who pioneered the neurosurgical treatment of epilepsy, was the first to start stimulating the brains of conscious humans. He wanted to be able to identify the parts of the brain involved in speech and movement, before he took out the piece of brain he thought was responsible for disease, so he developed ways to make a hole in the skull and expose the brain in fully conscious humans.
While he was in there stimulating the brain for clinical purposes, he also stimulated other parts of brain. He showed that by stimulating visual cortex, you can get people to see stars or flashes of light. When he stimulated the auditory cortex, people could hear buzzing signals. When he went deeper into the brain, into the temporal cortex, he could elicit complex perceptions. A patient would say things like, 'I'm sitting on the back porch of my mother's house and she's calling me to dinner.'
He did all of this in the 1930s, but the field never went anywhere because he knew nothing about the circuitry of the brain. Penfield was just stimulating neural tissue of an unknown nature. He could elicit conscious phenomena, but he gained no insight into how, exactly, the conscious phenomena are related to the [behavior] of the activated neurons.
Now we know about single cells, neural circuits, and their selective properties. So we can make better hypotheses about how cells might contribute to cognitive phenomena such as perception or memory or attention. We can tweak carefully targeted parts of the system and get a predictable response.
TR: So how do you plan to understand the link between activity in specific parts of the brain and consciousness?
BN: I don't now how to figure it out, but it seems to me that stimulating a human brain such as my own would be a good place to start. If I could stimulate my MT, then, presumably I would know and could say whether I really see the [actual] dots moving [as in the monkey experiments] or something else altogether. This would be a start toward identifying the [specific aspects of consciousness that accompany] neural activation at different points in the nervous system.
TR: Do you think you could really get regulatory approval? What are the major ethical issues?
BN: Getting approval to do something like this would be difficult. Any human experiments in this country are under rigorous scrutiny. Lawyers and administrators at institutions take a dim view of this kind of thing because of the liability issues. And there is a definite slippery slope argument. I might be able to make a case for my own experiment, but it could set precedent for others for whom it would be more risky.
For example, if I did this experiment, it would probably be a big deal and get in the newspapers. Some young graduate student might see it as a way to get ahead in his career and decide to do it. He might put himself at greater risk than I would. Maybe he would probe deeper into his brain, where there is more risk of damaging the vasculature. It would be uncomfortable to think that I was responsible in part for that, even if my own adventure turned out just fine.
TR: Do you really want to do this?
BN: Well, I've thought about it very carefully. I've talked to neurosurgeons, both in the United States and outside the country where the regulatory environment is less strict, about how practical and risky it is. If the risk of serious postsurgical complications was one in one hundred, I wouldn't do it. If it was one in one thousand, I would seriously consider doing it. To my chagrin, most surgeons estimate the risk to be somewhere in between my benchmarks.
TR: Most of your experiments have been done on monkeys. How did that begin to shape your view on the relationship between brain functions and human consciousness?
BN: We study motion perception. We train monkeys to look at a pattern of dots moving in a certain direction and to report the direction of the dots by moving their eyes in the same direction. If a monkey picks the correct answer, he gets a reward.
This simple behavior contains a world in terms of understanding how the nervous system performs intelligent behavior. Sensory information that comes into the brain through the eye must be coded into some neural language that represents the stimulus within the brain. Based on this neural representation, the monkey must then make a high-level judgment about what he is actually seeing. This "decision" in turn guides the selection of a motor response, to look to the left or the right.
TR: And you added a new level to this experimental setup by stimulating the monkey's brain.
BN: We put an electrode in an area of the brain known as MT. The cells in this area respond selectively to a specific direction of motion. Some cells are active when the monkey looks at dots moving to the left, some cells are active when the monkey looks at dots moving to the right. People had suspected for a long time that MT was important for our ability to see motion. So we did an experiment where we stimulated these cells artificially with tiny pulses of electrical current -- it changed what the monkeys reported seeing.
TR: So with the monkey experiments, you can stimulate the brain in very focused ways and change the way the monkey responds. But the monkey can't tell you what he sees when you stimulate the brain.
BN: Yes. People can report what they see or hear or feel, but with monkeys, you can only look at their change in behavior. I can't climb into a monkey's head and see what the monkey really sees.
This gets to core of the current debate about the study of consciousness. What is the conscious experience that accompanies the stimulation and the monkey's decision? Even if you knew everything about how the neurons encode and transmit information, you may not know what the monkey experiences when we stimulate his MT.
TR: People have shown that stimulating the human brain can do similar things too, right?
BN: Electrical stimulation of the brain is not new. Wilder Penfield, a neurosurgeon in Canada in 1930s and 40s, who pioneered the neurosurgical treatment of epilepsy, was the first to start stimulating the brains of conscious humans. He wanted to be able to identify the parts of the brain involved in speech and movement, before he took out the piece of brain he thought was responsible for disease, so he developed ways to make a hole in the skull and expose the brain in fully conscious humans.
While he was in there stimulating the brain for clinical purposes, he also stimulated other parts of brain. He showed that by stimulating visual cortex, you can get people to see stars or flashes of light. When he stimulated the auditory cortex, people could hear buzzing signals. When he went deeper into the brain, into the temporal cortex, he could elicit complex perceptions. A patient would say things like, 'I'm sitting on the back porch of my mother's house and she's calling me to dinner.'
He did all of this in the 1930s, but the field never went anywhere because he knew nothing about the circuitry of the brain. Penfield was just stimulating neural tissue of an unknown nature. He could elicit conscious phenomena, but he gained no insight into how, exactly, the conscious phenomena are related to the [behavior] of the activated neurons.
Now we know about single cells, neural circuits, and their selective properties. So we can make better hypotheses about how cells might contribute to cognitive phenomena such as perception or memory or attention. We can tweak carefully targeted parts of the system and get a predictable response.
TR: So how do you plan to understand the link between activity in specific parts of the brain and consciousness?
BN: I don't now how to figure it out, but it seems to me that stimulating a human brain such as my own would be a good place to start. If I could stimulate my MT, then, presumably I would know and could say whether I really see the [actual] dots moving [as in the monkey experiments] or something else altogether. This would be a start toward identifying the [specific aspects of consciousness that accompany] neural activation at different points in the nervous system.
TR: Do you think you could really get regulatory approval? What are the major ethical issues?
BN: Getting approval to do something like this would be difficult. Any human experiments in this country are under rigorous scrutiny. Lawyers and administrators at institutions take a dim view of this kind of thing because of the liability issues. And there is a definite slippery slope argument. I might be able to make a case for my own experiment, but it could set precedent for others for whom it would be more risky.
For example, if I did this experiment, it would probably be a big deal and get in the newspapers. Some young graduate student might see it as a way to get ahead in his career and decide to do it. He might put himself at greater risk than I would. Maybe he would probe deeper into his brain, where there is more risk of damaging the vasculature. It would be uncomfortable to think that I was responsible in part for that, even if my own adventure turned out just fine.
TR: Do you really want to do this?
BN: Well, I've thought about it very carefully. I've talked to neurosurgeons, both in the United States and outside the country where the regulatory environment is less strict, about how practical and risky it is. If the risk of serious postsurgical complications was one in one hundred, I wouldn't do it. If it was one in one thousand, I would seriously consider doing it. To my chagrin, most surgeons estimate the risk to be somewhere in between my benchmarks.
source:http://www.technologyreview.com/BioTech-Devices/wtr_16325,306,p1.html?PM=GO
Majestic Research finds favor with hedge funds
NEW YORK, Feb 14 (Reuters) - Stock tracker Majestic Research doesn't talk to company executives or visit malls to get a handle on retail trends. Nor does it make stock recommendations.
But the New York-based research firm is winning converts among hedge funds who say its brand of Web-based quantitative analysis can be more accurate than traditional Wall Street research forecasts.
With e-commerce now comprising a larger share of consumer spending than ever before, Web-based data mining has become a rich source to track products sales and generate sales leads.
But instead of sales leads, Majestic uses Web data to track sales and forecast financial results for target companies - ahead of quarterly earnings releases.
"It's really been an invaluable service," said Isabelle Fymat, partner in the $1.2 billion hedge fund Crosslink Capital, referring to Majestic. Fymat, who tracks e-commerce stocks like Yahoo (YHOO.O: Quote, Profile, Research) and Amazon.com (AMZN.O: Quote, Profile, Research), said Majestic supplies "a lot more reliable information on what is driving or hurting revenue."
Majestic, which was founded in 2002, uses "quantitative" analysis that it claims can do the job better than traditional stock research methods, at least for consumer-sensitive companies that utilize the Internet in some way.
From modestly-furnished Majestic offices overlooking Manhattan's Central Park, several dozen math Ph.D.s, statisticians and other quantitative analysts evaluate data spewed from computers using "Web crawling" programs track sales and other information from tens of millions of Web pages or other on-line resources.
The information is used to get a better and more timely picture of sales for roughly 60 companies Majestic tracks, including Yahoo, Carmax (KMX.N: Quote, Profile, Research), JetBlue (JBLU.O: Quote, Profile, Research) and Carnival Corp. (CCL.N: Quote, Profile, Research).
The firm's methods differ from traditional Wall Street research, where analysts make forecasts based on conversations with company executives, advertisers, suppliers and mall visits to forecast company results and make recommendations.
"This firm did to research what Instinet (INGP.O: Quote, Profile, Research) did to trading," said Majestic chief executive Doug Atkin, referring to the electronic stock exchange he headed for a decade prior to joining Majestic. "We offer very differentiated research."
While some might dismiss such comments as hyperbole, some hedge funds say Majestic's analysis gives them a proprietary edge in a hyper-competitive hedge fund world. Wall Street investment banks are skimping on stock research in the wake of a series of regulatory probes, making it of limited value to active traders like hedge funds, they say.
"The Street is not paid for research anymore and the quality of research is diminished greatly," said Dan Frank, portfolio manager for Cerberus Capital Management, a $16 billion hedge, buyout and lending firm.
Frank said he uses Majestic for its drug prescription sales data generated from some 3,500 physicians. With that information, he can get a sense of how new drugs are selling and predict stock performance for drug stocks before they disclose quarterly performance.
"It's just raw data," said Frank. "I don't want their opinion on stocks. I want particular data on a particular product."
Still, even with access to billions of bits of information, Majestic isn't always right. Majestic was off on forecasts for Google's fourth-quarter results last month, along with most other research firms, according to Crosslink's Fymat.
"It's not 100 percent accurate," said Fymat. "But historically they have been very very close. And it's way better than Wall Street research."
Some experts said hedge funds won't ever replace Wall Street research with quantitative methods developed in-house or provided by Majestic. But fund managers can be quick to adopt new kinds of research, said Justin Dew, senior hedge fund analyst at Standard & Poor's.
"Conceptually, there is always a need for whatever is the next cutting edge in identifying inefficiencies in markets," said Dew. "Hedge funds tend to be first or early adopters."
source:http://today.reuters.com/investing/financeArticle.aspx?type=fundsFundsNews&storyID=2006-02-14T191858Z_01_N14387237_RTRIDST_0_FINANCIAL-MAJESTIC-HEDGE.XML