Monday, November 03, 2014

The future of intelligence is distributed - and so is the future of government

In 2011 an IBM computer, Watson, beat human competitors at Jeopardy! 

This was a new landmark in artificial intelligence - a computer capable of correctly responding to plain English questions, in real time, by figuring out their intent.

At the time Watson was a computer as big as a room, and it was the only one of its kind in the world.

The original Watson still exists, as discussed in this Wired article, The Three Breakthroughs That Have Finally Unleashed AI on the World, however it is no longer alone.

Hundreds of Watsons are now in operation - not as room-sized computers, but operating 'in the cloud', as distributed software across thousands of open-source servers.

People can access the intellect and computing power of these Watsons through any computing device connected to the internet.

Even more significantly, like many artificial intelligences, Watson is a learning machine that gets more knowledgeable and able to find insights the more it learns. Whenever a Watson learns something, making a new connection, that knowledge is shared with every Watson - making it a distributed intelligence, able to learn at rates far faster than even a single supercomputer, or human, is able to learn.

The power of Watson isn't in the revolutionary algorithms that power its learning, it's in the network itself - how separate Watsons can share knowledge and learn from each other.

This is how humans evolved civilisation - by capturing, codifying, storing and sharing knowledge in sounds, images and words to pass it on from one individual to another.

However Watson hints at a more robust future for human intelligence, and for how we govern ourselves.

Humans have proven over the centuries that having more learners with better knowledge sharing means faster progress and better decision-making. Books, universal schooling and the internet have shown how dramatically a society can progress when appropriate knowledge sharing systems are in place.

The key is to focus on the size and complexity of the networks, not the expertise of individual 'nodes' (you might call them humans).

For computers this means that the more Watsons we create, and the more complex the knowledge sharing between them, the faster they will learn.

For governments this means the greater the transparency, and the more informed citizens are participating in knowledge sharing, the better the decisions and outcomes will be.

Now this isn't how government is currently constituted. The notion of representative democracy is that governance is handed to experts and specialists who live and breathe government so the rest of the population doesn't have to.

We elect politicians who are supposed to representative the interests of their electorates, and appoint bureaucrats whose role is to provide specialist knowledge and operate the machinery of government - develop policy, design and deliver programs, enforce laws and support citizens in emergencies.

By its nature this approach to government relies on experts who are placed separately to the population - often even physically removed and concentrated in a city like Canberra, Washington, Ottawa, Brazilia, Naypyidaw or Putrajaya.

This group (elected and appointed public servants alike) tend to become inwards focused - focused on how to make government keep working, not on whether it actually works and delivers for citizens.

Particularly inwardly focused governments tend to become so removed from their citizens that they are overthrown - though they've usually replaced with a not-dissimilar system.

Now we can do much better.

Rather than focusing on electing and appointing individual experts - the 'nodes' in our governance system, governments need to focus on the network that interconnects citizens, government, business, not-for-profits and other entities.

Rather than limiting decision making to a small core of elected officials (supported by appointed and self-nominated 'experts'), we need to design decision-making systems which empower broad groups of citizens to self-inform and involve themselves at appropriate steps of decision-making processes.

This isn't quite direct democracy - where the population weighs in on every issue, but it certainly is a few steps removed from the alienating 'representative democracy' that many countries use today.

What this model of governance allows for is far more agile and iterative policy debates, rapid testing and improvement of programs and managed distributed community support - where anyone in a community can offer to help others within a framework which values, supports and rewards their involvement, rather than looks at it with suspicion and places many barriers in the way.

Of course we need the mechanisms designed to support this model of government, and the notion that they will simply evolve out of our existing system is quite naive.

Our current governance structures are evolutionary - based on the principle that better approaches will beat out ineffective and inefficient ones. Both history and animal evolution have shown that inefficient organisms can survive for extremely long times, and can require radical environmental change (such as mass extinction events) for new forms to be successful.

On top of this the evolution of government is particularly slow as there's far fewer connections between the 200-odd national governments in the world than between the 200+ Watson artificial intelligences in the world.

While every Watson learns what other Watsons learn rapidly, governments have stilted and formal mechanisms for connection that mean that it can take decades - or even longer - for them to recognise successes and failures in others. 

In other words, while we have a diverse group of governments all attempting to solve many of the same basic problems, the network effect isn't working as they are all too inward focused and have focused on developing expertise 'nodes' (individuals) rather than expert networks (connections).

This isn't something that can be fixed by one, or even a group of ten or more governments - thereby leaving humanity in the position of having to repeat the same errors time and time again, approving the same drugs, testing the same welfare systems, trialing the same legal regimes, even when we have examples of their failures and successes we could be learning from.

So therefore the best solution - perhaps the only workable solution for the likely duration of human civilisation on this planet - is to do what some of our forefather did and design new forms of government in a planned way.

Rather than letting governments slowly and haphazardly evolve through trial and error, we should take a leaf out of the book of engineers, and place a concerted effort into designing governance systems that meet human needs.

These systems should involve and nurture strong networks, focusing on the connections rather than the nodes - allowing us to both leverage the full capabilities of society in its own betterment and to rapidly adjust settings when environments and needs change.

We managed to design our way from the primitive and basic computers of the 1950s to distributed artificial intelligences in less than 70 years.

What could we do if we placed the same resources and attention on designing governance systems that suited modern society's needs?

And it all comes down to applying a distributed model to governance - both its design and its operation, rather than focusing on the elevation of individual experts and leaders to rule over us.

It's a big challenge, but for a species that went from horses to spaceships in two generations, it surely isn't an impossible one.

And given that societies thrive or die depending on how they are governed, are we willing to take the the risk and hope that our current governance and political systems simple evolve into more effective forms within a human lifespan?

Read full post...

Wednesday, October 22, 2014

Use of government open data creeping into universities

In September I was honoured to be invited to speak at the first Best Practice in Data Journalism seminar for Australia at Melbourne University, hosted by the Centre for Advancing Journalism.

The event included many of Australia's leading data journalists, together with academics and other representatives working in the data space.

At the event I was glad to see that a number of Masters students in the university were using government open data to create visualisations and derive insights.

Richard Sinnott has now published a public list of some of these projects in the Asia-Pacific data journalism and data visualisation group at LinkedIn, which I've included below.

These have been been developed in ten weeks or less and are works in progress, so consider them appropriately.

Federal government open data mash-up:

Visualising Victorian lobbyists, clients, companies, donors:
(wait for it to load as there is a large complex graph).
Hopefully straightforward to drive/use...

Using Twitter for Melbourne transport congestion:
(final scenarios coming soon)

Using Twitter for World Cup 2014:
(can we identify events in games through Twitter sentiment) - pretty much yes as long as enough goals scored!
Also finalising scenarios on are Italians really more emotional than Germans (sentiment +/- on Twitter!)

What politicians are tweeting in Victoria:

What politicians are tweeting in Federal government:

Read full post...

Friday, October 10, 2014

Don't blame the technology, blame the humans

This week one of the top stories for one of Australia's leading newspapers has been the purported rape of a woman after she met a man using the 'Tinder' matchmaking app.

Rape is a terrible act and unacceptable in every circumstance.

However the newspaper chose to make Tinder the story, not the rape.

Tinder is a mobile application designed to help people meet prospective dates.

It simply alerts users to other Tinder users who fall within a specified age range and gender and are within a certain distance of your location, lettings you know whether you have any mutual friends.

The alerted user then decides whether they like the look of a person from a supplied photo and gives them the option to privately message, and if desired, hook up.

Users must actively choose to turn on Tinder and can reject others without the other user ever knowing.

In other words, Tinder isn't much different to visiting a bar and looking around to see who seems to be your age and your 'type'.

Tinder's popularity stems from its ability to allow users to be discreet when seeking a partner. It gives users control over their dating choices - when they are available, who can be matched to them and whether or not they wish to be approached. In other words it's a little better than hanging around in pubs and nightclubs hoping to meet the perfect partner and avoid undesired come-ons.

In many senses the app is no different to popular online dating websites like HeyCupid and RSVP. it is simply a technological tool adding convenience and control to matchmaking.

Like all other technologies and software Tinder is neither good nor bad, but simply meets a human need.

So when a newspaper uses headline like 'Police warning after Tinder date ends in gang rape' and
'Tinder 'gang-rape victim' withdraws statement' it really gets my back up.

The approach creates an inappropriate connection between a neutral technology and a disgusting human act.

If the lady in question had simply met her alleged attackers in a pub, the newspaper would likely have not reported the story, or reported it very differently - they would not have used a headline 'Police warning after pub date ends in gang rape'.

This type of reporting is part of what holds back the use of digital technologies by government agencies, and it is a damn shame.

When senior managers who don't use social tools only read, hear and see bad news articles which blame or associate specific technologies with human misconduct it can creates an inappropriate association and make digital seems more dangerous to use than it actually is.

I wish I had a dollar for every media story sensationalising the failings of Facebook, Twitter, Google, RSVP, Tinder and other social tools when the real failing is in the human users.

Yes there are risks to all technologies. Cars and other road vehicles kill over 1,000 Australians each year and many other people are hurt when using other technologies, from paper cuts to knife wounds.

However let's try to keep things in proportion. There's few technologies out there specifically designed to harm people and these are usually carefully controlled, like guns and explosives.

The vast majority of technology are neutral, able to be used for good or bad - harming humans through accident or deliberate misuse.

Mitigating these risks is possible. Obsessing over them is unnecessary.

So if you're ever confronted in the workplace by a colleague or manager who quotes a headline like Police warning after Tinder date ends in gang rape' agree with them that the event is tragic and horrible.

But remind them that it was a human act that made it tragic and horrible. It is not necessarily a fault of the technological tool.

Read full post...

Thursday, October 09, 2014

IP Australia releases open data including over 100 years of patent, trademark, design and plant breeder rights records

In a great step forward for Australian open data, IP Australia has, for the first time in history, released over 100 years of patent, trademark, design and plant breeder rights records as free publicly accessible and machine-readable data.

Released through (at, while not a real-time feed, the information contained on Australia's innovation history is staggering.

Note that the main data file is 430Mb as a zip, it can take some time to download and process.

It will be interesting to see how this data might be used over the next year and particularly in future hack competitions, such as GovHack 2015.

Read full post...

Wednesday, October 08, 2014

How current events play out in search requests - terrorism & related terms in Google trends

While agencies often invest significant money into tools for tracking trends on social media, one of the simplest ways to detect and monitor the rise and fall of key topics and issues online is through Google Trends.

Google Trends tracks the frequency of use of specific search words in Google searches. This represents the majority of online and mobile searches in countries like Australia (93%) and the US (68%)

As a free service, Google Trends has been used over the years to monitor trends in seasonal diseases, such as influenza and dengue fever, to track the relative level of attention paid to politicians, the number of mentions of sports during grand final seasons, and to understand the impact of advertising on product sales.

I used the service back in 2006-2007 to help track a government agency's rebranding program, and have used it subsequently, both with and outside government, to track the level of interest in particular issues and topics.

So today I decided to see what Google Trends can tell us about the level of interest or concern in terrorism, specifically related to ISIS and concerns about muslim extremists.

I chose five main words to track - 'Terrorist', 'ISIS', 'Islam', 'Muslim' and 'Burqa' - which told an interesting story.

Until May 2010 the burqa does not appear to be a particular concern for Australians, with few searches of the term.

However since then it has become more topical, with some interest throughout 2011, then a sudden surge in September 2014 when the 'ban the burqa' movement began to receive significant political support and media coverage.

In contrast, terrorist was a term of interests to Australians in 2004 and particularly in the second half of 2005, with surging interest in July and November of that year. Following this, it settled down into a largely quiescent state, with only a small surge in November 2008 interrupting the mostly flat line.

This changed in August 2014, with a huge rise in searches for the term across Australia resulting in the highest level of searches for the term in the history of Google Trends in September this year.

The same trend can be seen for mentions of ISIS, which were flat until May 2014 and have rapidly escalated since. Early mentions of the term presumably relate to other uses of the term (such as the Egyptian god), with the sudden rise in searches only attributable to the rise of the Islamic State.

Searches for Islam and Muslim have also been rising this year after a long largely flat period. While these terms are the subject of many legitimate searches related to the culture and religion, the recent rise in searches does tend to suggest and correlate with the rise in searches for terror-related terms, indicating that people have linked the terms in some way, at least out of curiosity.

It's possible to compare and contrast these trends with global trends in Google Trends, per the chart below.

This chart provides evidence of growing global interest in terms such as Islam, Muslim and, particularly, ISIS. However it shows little international concern over the burqa or regarding terrorism.

This can be seen in detail when looking at individual countries.

For example while similar trends of increased interest in searching the term ISIS are visible for the USUK, Canada, SwedenJapan, Thailand and many others, only a relative few see the burqa as a rising source of concern and many also are not experiencing heightened searches for terms such as islam or muslim.

This may be coincidental, or may reflect political statements and media reports on these topics - a more detailed review of coverage would be needed to confirm direct links.

However given that researchers have found that Google Trends can provide an accurate view of community concerns regarding infectious diseases and product trends, I believe there's sound reasons to suppose a correlation between what leaders say and what people search for.

Read full post...