Showing posts with label open data. Show all posts
Showing posts with label open data. Show all posts

Thursday, February 02, 2023

It's time for Australian government to take artificial intelligence (AI) seriously

Over the last two and a half years I've been deep in a startup using generative artificial intelligence (AI that writes text) to help solve the challenge organisations face in producing and consuming useful content.

This has given me practical insights into the state of the AI industry and how AI technologies can be successfully - or unsuccessfully - implemented within organisations to solve common challenges in the production, repurposing and reuse of content.

So, with a little prompting from the formidable Pia Andrews, I'm taking up blogging again at eGovAU to share some of my experience and insights for government use of AI.

I realise that Australian governments are not new to AI. Many agencies have been using various forms of AI technologies, directly or indirectly, to assist in understanding data or make decisions. 

Some may even include RPA (Robotic Process Automation) and chatbots - which in my humble opinion are not true AI, as they both are designed programmatically and cannot offer insights or resolve problems outside their programmed parameters and intents.

When I talk about AI, my focus is on systems based on machine-learning, where the AI was built from a body of training, evolving its own understanding of context, patterns and relationships.

These 'thinking' machines are capable of leaps of logic (and illogic) beyond any programmed system, which makes them ideal in situations where there are many edge cases, some of which can't be easily predicted or prepared for. It also places them much closer to being general intelligences, and they often exhibit valuable emergent talents alongside their original reasons for being. 

At the same time machine-learning is unsuitable for situations where a decision must be completely explainable. Like humans it is very hard to fully understand how a machine-learning algorithm came to a given conclusion or decision.

As such their utility is not in the realm of automated decision-making, but rather is assistive by encapsulating an evidence base or surfacing details in large datasets that humans might overlook.

As such machine-learning has vast utility for government. 

For example,

  • summarizing reports, 
  • converting complex language into plain, 
  • writing draft minutes from an intended purpose and evidence-base, 
  • extracting insights and conclusions from large research/consultation sets, 
  • crafting hundreds of variants to a message for different audiences and mediums,
  • developing structured strategy and communication plans from unstructured notes,
  • writing and updating policies and tender requests, 
  • semantically mapping and summarizing consultation responses,
  • developing programming code, and
  • assisting in all forms of unstructured engagement and information summarization/repurposing.

As such machine-learning is as an assistive and augmentation tool. Extending the capabilities of humans by doing the heavy lifting, rather than fully automating processes.

It's also critical to recognise that AI of this type isn't the sole purview of IT professionals and data scientists. Working with natural language AIs, as I do, is better supported by a strong business and communications skillset than by programming expertise. 

Designing prompts for an AI (the statements and questions that tell the AI what you want) requires an excellent grasp of language nuances and an extensive vocabulary.

Finetuning these AIs requires a strong understanding of the context of information and what constitutes bias, so that an AI is not inadvertently trained to form unwanted patterns and derive irrelevant or unneeded insights.

These are skills that 'business' folks in government agencies often possess to a far greater degree than most IT teams.


So through my eGovAU blog, I'm going to be regularly covering some of the opportunities and challenges I see for governments in Australia seeking to adopt AI (the machine-learning kind), and initiatives I see other governments adopting.

I will also blog occasionally on other eGov (or digital government) topics, however as this is now well-embedded in government, I'll only do so when there's something new I have to add.

Read full post...

Monday, November 12, 2018

The #GovHack 2018 National Awards by the numbers

I shared this via a Twitter thread, but wanted to include it here for longevity.

To learn more about GovHack and the National Red Carpet event, which I attended as a representative of my team & the ACT Spirit of GovHack winner (and finalist for the National Spirit of GovHack), visit www.govhack.org

    




I've analysed the #opendata & here's the #GovHack 2018 Awards by the numbers:

There were 33 National Awards (including Spirit & Government Participation).
A total 88 Awards were issued: 33 First places, 18 Runner-ups and 37 Hon. Mentions.

Excluding Spirit of #GovHack & Gov Participation 59 teams won at least one Award.

Two teams won 3 Awards each:
  • Tiny Happy People Hacking: 1 First place, 1 Runner-up, 1 Hon. Mention
  • in time: 2 Runner-ups & 1 Hon. Mention
(Incidentally Tiny Happy People Hacking was my team)

Another 16 teams won two #GovHack Awards.
  • 5 teams won 2 First places! (as.numeric, Blockheads, Tartans-AU, insolvit & TeamTeam)
  • Another 5 won 1 First place, with 3 also winning a Runner-up (Big Orange Brain, Oakton, TechPreppers) and 2 also an Hon. Mention (DataCake & TeamX).

41 #GovHack teams won 1 award: 13 won a First, 5 a Runners-up & 23 a Hon. Mention.
  • Firsts:
    ARVIS, Bachmanns and Fulwoods, Get Active USC, Hack aPEEL, I’m Learnding, Living Spirit, Lucky Shot, Motley Crue, Team Marika, Team Rocket, Technotelecomnicon, The Ogrelords, This Place

By state/territory, inc. State/Local Government Participation & Spirit of #GovHack, National #GovHack Awards followed population size (except ACT which punched above its weight):
  • NSW won 21 
  • Vic won 19
  • Qld won 17
  • ACT won 14
  • SA won 9
  • WA won 4
  • NT won 2
  • Tas won 0 (sorry folks)

The #GovHack results looks a little different in detail, with Victoria winning more First places than anyone else & Queensland tying with NSW! (the NA are the National Government Participation Awards which I excluded as they give ACT an unfair bonus)


In fact, using a 3-2-1 scoring system for First Place, Runner-up and Hon. Mention, Victoria outscores NSW, and ACT even closer to the top three in the raw #GovHack award numbers.


Finally, looking at #GovHack National Awards by venue, the central city venues did better than regional locations in all cases, except in Queensland - where the Sunshine Coast won more awards than anywhere else in Qld, including Brisbane... Amazing work guys!!!



And here's the table of National #GovHack Awards by venue...


And that's the wrap on the by the numbers Awards for #GovHack 2018.
My data is still a bit messy, but I'll clean it up and put it in a Google sheet at some point in the next week so others can access it.


Read full post...

Tuesday, August 01, 2017

Roundup from GovHack 2017

Starting in a single Canberra venue in 2009, GovHack is now the largest open data hacking competition for government worldwide, with over 3,000 participants, coaches, mentors and organisers across 36 venues around Australia and New Zealand.

Over a 46-hour period participants including coders, creatives, data crunchers and facilitators, redesign and reimagine citizen services and use open data to visualise fresh insights into government decision-making, taking part in a competition with over 80 prizes and a prize pool of over $250,000.

The event is organised and run by volunteers, but GovHack has support from the Australian and New Zealand Governments, all Australian state and territory governments and many local governments across ANZ, as well as a range of corporate sponsors. This was the first year that the Northern Territory became involved with the event.

Many senior public servants drop into the event over the weekend, and have a keen interest in using ideas from GovHack within their agencies.

This year Accenture was the Platinum Sponsor for GovHack, the first time a corporation has taken such a significant interest in the event - a trend I hope continues as these types of event gain steam as a creative way for companies and governments to innovate quickly.

Accenture sponsored two awards, the ‘Into the New’ award for Australia challenged participants to demonstrate innovation and new thinking in all forms. This could be new ways to experience and interact with public data or new approaches to citizen experiences that help citizen and governments journey into the new together. It attracted 138 entrants from around Australia, from a total of 373 projects submitted.

Accenture’s ‘Re:Invention’ award for New Zealand challenged participants to design a citizen experience that builds on something government already does to deliver a more effective and engaging way of interacting. It attracted 12 entrants from Wellington, Auckland and Hamilton, from a total of 66 New Zealand projects submitted.


GovHack by the numbers
While GovHack itself is over for 2017, state award events will be held in August, and an international Red Carpet event for National and International Award winners in October. You can view the closing video from GovHack 2017 here.

All the projects created this year are online in the GovHack Hackerspace, available for inspiration and learning – remaining online to provide hundreds of fresh perspectives on how government can deliver more value to citizens.

you can read more about GovHack 2017 in this LinkedIn post by a mentor, or on Twitter.

Read full post...

Wednesday, July 26, 2017

Get revved for GovHack across Australia & New Zealand (28-30 July)

As the world’s largest hackathon, GovHack  is on at over 25 locations across Australia and New Zealand again this year from Friday 6pm this week until Sunday afternoon (28-30 July).
With over 3,000 participants and 437 completed projects in 2016, GovHack is an opportunity to develop prototypes of new services, visualisations and mashups with government open data and other datasets with the chance to be nationally recognised and win prizes at national, state and local levels.

Supported by all levels of Australian government, GovHack is not just for programmers. Some of the projects in previous years have included board games and jewelry (for instance 3D printed bracelets of climate data), alongside websites, mobile apps, wearable apps and APIs.

National awards are announced at a Red Carpet Event, which filled the PowerHouse Museum in Sydney in 2015 (the last one I attended).


While some people form teams before the event, you can also come along as a solo participant, or form a team on the day – providing an opportunity to rub shoulders with all kinds of talented people.

There’s still room to register for some venues if you want to participate.

I’m helping run the ACT local event this year, so will be onsite at Canberra Grammar all weekend. If you’re participating here, come and say hi!

For more information visit the GovHack website or read last year’s report.

Read full post...

Wednesday, December 07, 2016

Australian Government submits Australia's first open government National Action Plan

Over the last four years the Australian Government has engaged in a stop-start process towards becoming a full member of the international Open Government Partnership, which now counts 70 countries in its membership, plus is beginning to explore state-based membership as well.

The Open Government Partnership was founded five years ago by eight countries to foster open government around the world, providing a forum for countries to share their initiatives and make ambitious commitments to open up government, in various ways, to public scrutiny.

While Australia was invited to be a founding member, the then Government never quite got around to joining, and the process since then has been slow and torturous.

However in the last year, with renewed support from Prime Minister Turnbull, a consistent process has seen the goal of having Australia become a full and active member finally achieved (after a brief hiccup due to a national election campaign slowed it down).

I've blogged previously about several of the consultations and steps that have been taken in 2015-16 to progress towards this achievement, and have supported the process in various ways where I could, both formally and informally.

There's been a number of other people instrumental in finally arriving at this point, both significant movers and shakers within Australia's small civil society movement and within the public service and I congratulate all of them for this achievement.

While Australia's National Action Plan won't go far enough for some, and the consultation process was not as structured or inclusive as others would like, actually getting a plan at all has been a huge achievement given Australia has had five Prime Ministers and four governments in the last six years.

Also this is only the first National Action Plan. It lasts for two years and is expected to be renewed after that point.

Now that Australia has started this process, the goal should be for all participants to ensure that each new National Action Plan is more ambitious and inclusive than the last.

Read full post...

Monday, November 28, 2016

Guest post from Henry Sherrell on access to open data for effective policy development

Henry Sherrell is a former Australian Public Servant who now works in policy research at the Australian National University.

As a researcher, open data has become an important input into his work. As such I thought it worth sharing (with his permission) this post from his blog, On The Move, as an example of some of the difficulties researchers still face in accessing data from the Australian Government for important policy work.

It is notable that since Henry published his post, only four days ago, the legislation regarding Henry's policy work is going back to parliament - still with no modelling of its impact on affected communities or any real public understanding of the potential consequences.

I've reproduced Henry's post as a guest post below in full. You can also view Henry's post here in On The Move.

My battle with the Australian Border Force Act: A small, but worrying, example

There are hundreds of interesting questions to ask when someone moves from one country to another. For as long as I can remember, Australia has been one of the best places to explore migration. There are two reasons for this: We welcome immigrants and the government and bureaucracy collect and make accessible robust migration data.
They are not household names but people like Graeme Hugo, the late Paul Miller, Deborah Cobb-Clarke and Peter McDonald have shaped global debates on migration. A new generation of scholars are now examining big, important questions about the intersection migration and work as well as any number of other themes, many of which will help us as a society in the future. Yet this tradition depends on access to Australian migration data from a number of sources, including the ABS, the Department of Immigration and various surveys funded by the government.
Until I received the following email from DIBP, I hadn’t realised just how uncertain this type of knowledge will be in the future:
“The data that was provided to Department of Agriculture was done so for a specific purpose in line with the Australian Border Force Act 2015 (ABF Act).  Unfortunately your request does not comply with the ABF Act and we are therefore unable to provide the requested data.”
I didn’t receive this email because I asked for something controversial. The reason this email stopped me in my tracks was I asked for something which was already largely public.
About a month ago I stumbled across the below map in a Senate submission to the Working Holiday Reform legislation.  The Department of Agricultural and ABARES had produced the map to help show where backpackers worked to gain their second visa. This was an important part of a big public debate about the merits or otherwise of the backpacker tax (as I write this legislation has just been voted on in the Senate, amended and defeat for the government).
I’d never seen this information before and I’m interested in exploring it further as there are decent labour market implications stemming from backpackers and the results may shed light on employment and migration trends. As you can see below, the Department helpfully documented the top 10 postcodes where backpackers worked to become eligible for their 2nd visa:
screen-shot-2016-11-24-at-2-48-38-pm
I get teased a little bit about the number of emails I send asking for stuff. But I’ve found you normally don’t get something unless you ask for it. So using the Department of Agriculture’s handy feedback form on their website, I asked for the data showing how many 2nd working holiday visas have been granted for each postcode.
The top 10 postcodes are already public but as the map shows, there is lots of other information about what you might term a ‘long tail’ of postcodes. One reason I wanted this information was to match up major industries in these postcodes and understand what type of work these people were doing. It would also be good to go back a couple of years and compare trends over time, whether employment activity shifts over time. All sorts of things were possible.
One thing I’ve learnt in the past is don’t ask for too much, too soon. In addition, there is always a potential privacy consideration when examining immigration data. For these reasons, I limited my request to the list of postcodes and number of second visa grants in each. That’s it.
This ensured I excluded information about individuals like age and country of birth which may compromise privacy. I also assumed if the number of backpackers in a postcode was less than five, it would be shown as “<5 as="" data.="" for="" immigration="" is="" of="" other="" p="" practice="" standard="" this="" types="">
ABARES let me know they had passed the response to the Department of Immigration and Border Protection. After following up with DIBP twice, about a month after my initial request, I received the above email which prompted a series of internal questions roughly in this order:
  • You have to be f****** kidding me?
  • If the data was provided to the Department of Agricultural with the knowledge it would be at least partially public, why isn’t the same data available but in a different format? i.e. a spreadsheet not a map based
  • How does my request not comply with the ABF Act? What’s in the ABF Act which prevents highly aggregated data being shared to better inform our understanding of relevant public debates?
And finally: why couldn’t someone work out a way to comply with the ABF Act and still provide me with data?
From what I can work out, the relevant part of the ABF Act is Part 6 pertaining to secrecy and disclosure provisions. Section 44 outlines ‘Disclosure to certain bodies and persons’ and subsection (1) is about ‘protected information that is not personal information’ disclosed to “an entrusted person”. This is the same process causing serious consternation among health professionals working in detention centres.
I am not “an entrusted person”. According to subsection (3), the Secretary of the Department has authority to designate this. Perhaps I should email and ask? Again from what I can work out, it looks like the person who created the data made a record now classified as protected information. This information is then automatically restricted to people who are classified as entrusted, including other bureaucrats, such as those in the Department of Agriculture.
Yet this begs the question. If the Department of Agriculture can publish a partial piece of a protected record, why can’t the Department of Immigration and Border Protection?
All I know is this stinks. And while this concern does not rank anywhere close to those faced by doctors and nurses who work in detention centres, the slow corrosion of sharing information caused directly by this legislation will have massive costs to how we understand migration in Australia.
Think about the very reason we’re even having a debate about the backpacker tax. Not enough people knew about immigration policy, trends and behaviour. The wonks at Treasury didn’t do any modelling on the labour market implications and the politicians in ERC and Cabinet – including the National Party – had no idea about what this might do to their own constituents. Outside the government, when I did a quick ring around in the days after the 2015 budget, the peak industry groups for horticultural didn’t think the backpacker tax would be a big deal. If I was a farmer, I’d rip up my membership. People should have known from very early on this would have real effects in the labour market as I wrote 10 days after the Budget. The fact no-one stopped or modified the tax before it got out of control shows we are working off a low base in terms of awareness about immigration.
The Australian Border Force Act is only going to make that more difficult. Hiding basic, aggregated data behind this legislation will increase future episodes of poor policy making and limit the ability of Australia to set an example to world for immigration. Our Prime Minister is fond of musing on our successful multicultural society yet alongside this decades of learning that has shaped communities, policy decisions, funding allocations and everything else under the sun.
I have no idea how I’m meant to take part in this process if access to information is restricted to bureaucrats and ‘entrusted persons’, who at the moment don’t seem able to analyse worth a damn, judging from the quality of public debates we are having. I don’t expect a personalised service with open access to immigration data. But I expect the public service to serve the public interest, especially when the matter is straightforward, uncontroversial and has the potential to inform relevant public debate.

Read full post...

Thursday, November 24, 2016

Census 2016 Senate Inquiry report - what's been recommended to avoid another #CensusFail

Both the Senate Inquiry report on Census 2016 and the Review of the Events Surrounding the 2016 eCensus (by Alastair MacGibbon, Special Adviser to the Prime Minister on Cyber Security) have just been released - I've embedded both reports below (so they appear in one place).

They are a good read with some useful recommendations for the future.

Reflecting on what has become known as #CensusFail, in comparison to other technical issues experienced by government, the Census 2016 experience probably rates as the most significant public ICT issue experienced by the Australian Government so far this century.

While in the last 15 years the Australian Government has had other incidents, these have been relatively minor, with limited public visibility or impact.

This includes incidents such as the 15 year delay in creating an Apple version of e-Tax (now rectified), launch issues with sites such as MySchool, challenges with access and security within the MyGov system, data breaches from the PCEHR (personally controlled electronic health records) eHealth system and the accidental exposure of private data for asylum seekers.

In contrast, the issues experienced during the 2016 Census have been far more widespread in their public visibility, impact and long-term ramifications for trust in government.

However, to put "the most significant public ICT issue experienced by the Australian Government this so far this century" into perspective - no-one died, no-one was hurt and no-one even lost control of their personal data.

At worst a number of government and IBM staff experienced unhealthy levels of anxiety for several days.

Given the struggles that developing countries have had to get their egovernment ICT working in the first place (with a reported 15% success rate); or the challenges advanced countries like the US have had with national systems (such as ObamaCare); or the experience of states like Queensland, which could not pay some of its Health staff for some time when its new payroll system failed, CensusFail just doesn't rate as an ICT disaster.

The actual operational impact of the 2016 Census problems was merely a short delay for people attempting to fill in the Census online.

Ultimately the ABS still exceeded the desired Census response rate, will still be releasing Census data much faster than ever before, and the agency still saved over $70 million dollars by moving more of the Census online.

However despite not actually rating as a ICT disaster, there was still a real cost to CensusFail - the perceptual and reputational damage from the ABS publicly failing to deliver on its Census Night promise, exacerbated by poor crisis engagement.

As a net result the real impact of CensusFail is on long-term governance in Australian, due to a reduction in trust in public institutions to 'do the job right the first time'.

I'm aware of other agencies now being regularly questioned by their Ministerial offices on whether they have any systems or projects which pose a similar reputational risk to the Australian Government. I've watched as the term 'CensusFail' has become the 'go to' term raised whenever a new government ICT issue is reported.

As a result the trust in government agencies to deliver complex technical solutions has been diminished, and it will take years to recover.

I hope that the recommendations in this Senate report, the lessons from Census 2016, will be top-of-mind for every public servant and Minister engaged in a significant government ICT project for years to come.

Hopefully the right lessons will be learnt - that managing your communications and public engagement well when the ICT gets wonky is critical.

In fact you can even transform a technical failure into an engagement success, if you get your messaging and timing right - strengthening, rather than weakening, trust in government.

Census 2016 Senate Inquiry Report as redistributed by Craig Thomler on Scribd


Read full post...

Monday, November 07, 2016

It's time to provide feedback on Australia's Open Government Plan

Last week the government released Australia's draft Open Government National Action Plan, a requirement to join the international Open Government Partnership, for community consultation.

Australians have until the 18th November (Edit: extended from 14th based on community feedback) to comment on the plan, at which stage the government intends to move to rapidly endorse it via Cabinet and begin implementation.

The plan is available as a PDF download from the government's Open Government Partnership website (ogpau.govspace.gov.au) as well as in web format (much easier to read) at the civil society Open Government Partnership Network site (opengovernment.org.au).

I've included the 14 commitments proposed below in brief - for more detail click through to the sites. (Edit: thanks to Asher for corrected number).

Transparency and accountability in business

The Government will enhance Australia’s strong reputation for responsible, transparent and accountable business practice. 

Open data and digital transformation 

The Government will advance our commitments to make government data open by default and to digitally transform government services. 

Integrity in the public sector  

The Government will improve transparency and integrity in public sector activities to build public confidence and trust in government. 

Public Participation and Engagement

The Government will improve the way the Australian Government consults and engages with the Australian public.

Read full post...

Tuesday, August 30, 2016

Digital Transformation Office launches beta for their Digital Marketplace

Earlier this week the Digital Transformation Office (DTO) launched the beta version of their Digital Marketplace, a directory of vendors offering specialist digital services across a range of role categories.

The explicit reason for the Digital Marketplace was to make it easier for small and medium enterprises to engage with government, particular within large ICT projects. It is supposed to do this by allowing agencies to break down large projects into small stages which smaller companies are able to fulfil.

At this stage the Digital Marketplace is primarily a list of vendors - over 220. Most are small businesses, with a smattering of recruitment agencies (Horizon, Hudson, Randstadt, Talent International, The Recruitment Hive) and larger companies (such as Deloitte).

Right now it's possible for agencies to make both open and select requests to the list for skills via a briefing process, with additional approaches to market, such as an ideation approach both for buyers (roughly 'I have this problem, how would you solve it') and sellers (roughly 'I have this idea to solve a government problem - will anyone fund the work'), still under development.

The beta allows for fourteen role categories, covering a wide range of skills in the digital area, with more to come as the marketplace beds down and grows. The current roles are close to the DTO's core business of promoting and incubating digital transformation, which seems a reasonable place for them to start.

While the marketplace provides the information in a different way to most government procurement panels, it is governed in the same way - under a standing contract arrangement. At this stage all the innovation is at the front end and it will be interesting to see whether other agencies with whole-of-government panels (particularly Human Services and Immigration) see value in this way of displaying vendors and in the additional features the DTO plans for the site.

I've had a good look through the initial Digital Marketplace - in fact I'm affiliated with one of the participating vendors (as would be most private sector digital people in Canberra) - and it was interesting to see how many companies claim to have access to talent that government needs in the digital space.

Most government panels have been far more restrictive in the number of vendors they allow on the list, which has led to significant 'horse trading' of panel access and the development of services like SME Gateway to facilitate companies without a panel presence, whereas the DTO has gone for a 'bucket list' of any company that can demonstrate they meet the required criteria.

I've done a little analysis of the vendors in the Digital Marketplace and found a few interesting insights as to the responses the DTO received.

Firstly, half the approved vendors offer four or fewer of the fourteen role categories in the marketplace, with only 8% (generally recruitment companies) offering the full 14.


This suggests a lot of specialist providers have joined the service - companies which may otherwise struggle to meet procurement requirements without extensively partnering or contracting their services through larger providers.

The most popular role offered by vendors was Business Analyst, provided by 123 (or 55%), whereas the least popular was Ethical Hacker, provided by only 51 vendors (23%), followed by Inclusive Designer (Accessibility Consultant) by 58 (26%) of vendors.

This isn't surprising. Business Analyst is a standard role that has been around for a long time in ICT, whereas Ethical Hacker is relatively new as a role type and Accessibility remains an underrated area by government (with many practitioners struggling to find sufficient paying work).

It was interesting how many vendors offered personnel in the Digital Transformation Advisor role, which was second behind Business Analyst (113 vendors or 51%) despite being a very new role type.

I'm still sifting through the data and expect to find more interesting insights - particularly from the pricing (for which the DTO has published the ranges by role). This was an interesting decision by the DTO as it may encourage organisations to migrate pricing from below the given range upwards, and once in the range toward its top.

A lot of the data exposed in the marketplace has commercial significance, so I may not be able to share all of it, but the site is already gold for organisations seeking to understand the landscape servicing government. Couple the information in the site with industry knowledge and published tender amounts and it becomes relatively easy to identify the high and low price vendors.

Read full post...

Wednesday, August 03, 2016

The consequences of dropping the ball in digital engagement - The ABS and Australian Census 2016

Next week Australia will be holding its 17th national census (since 1911), led by the Australian Bureau of Statistics, which is itself celebrating its 110th anniversary as an agency (albeit with a name change midway).

This is an auspicious occasion for another reason. While it has been possible to complete the census online in both 2011 and 2006, when the ABS first trialled an online completion system - 2016 will mark the first occasion when the ABS expects a majority of households to complete their census surveys online.

In fact, Duncan Young, head of the 2016 Census process, is on record stating that the ABS expect 65% - two-thirds - of households to complete the Australian Census online, rather than in paper form.

This is a fantastic achievement and speaks highly to the ABS's commitment to quality data collection and maintaining a forward-facing approach to trialling and adopting new technologies.

This commitment has also been typified by the ABS BetaWorks Blog (sadly now defunct), ABS CodePlay (sadly not repeated) and the work the ABS has done to expose data in open and machine-readable formats, including ABS.Stat and APIs such as for the Population Clock.

Data collected by the ABS, particularly via the Australian Census, underpins an enormous amount of evidence-based decisions made by all levels of Australian government, as well as by companies who access the information to guide their commercial decisions.

The census is also an enormous undertaking. To quote Wikipedia quoting the 2011 Census site, "the 2011 Census was the largest logistical peacetime operation ever undertaken in Australia, employing over 43,000 field staff to ensure approximately 14.2 million forms were delivered to 9.8 million households." The cost was $440 million.

That makes the census a prime target for budget cuts - with the idea of reducing the frequency of the Australian Census to every ten years, or reducing its complexity, thrown around last year before being dropped.

The impact of not having regularly collected census data, collected in a compulsory manner from all households, can be hard for Australians to imagine.

However in countries like Lebanon, which hasn't had a census since 1932, the lack of accurate data leads to opinion-based government decision-making, which is generally viewed as a poor alternative to fact-based policy decisions.

The need for compulsory collection of census data was highlighted by the decision by the former Canadian government to make their long-form census voluntary in 2011, resulting in a massive drop in participation and corresponding degradation of data quality.

Called "a disaster for policy makers", unfortunately it suited the Canadian government of the day to not have accurate data in order to provide them greater room for making ideological decisions, rather than decisions that were based on facts. The net result was a drop in participation from 95% to 68%, a more expensive Census process (due to increased mailout of forms to prompt engagement), the resignation of several of the most experienced and competent senior officers in Canada's statistical agency, ongoing issues for national, provincial and local Canadian governments in identifying disadvantage, population numbers, statistical population changes and reduced capability for companies to make appropriate commercial decisions without investing in further expensive research.

The current Canadian government reinstated the compulsory long-form census, which completed collection in May this year.

So regular compulsory censuses are a BIG DEAL for a nation, and Australia has a very strong statistical foundation to build on.

The ABS has also demonstrated leadership in how it has marketed and communicated past Australian Censuses. In particular in 2011 the ABS demonstrated global leadership in the use of digital channels and tools to promote the importance of the Census and lift participation.

Through quirky best practice engagement on Twitter and Facebook, which made the Australian Census front-page news for all the right reasons, the development of an interactive online service allowing people to 'place' themselves within Australia, and a mobile game which allowed people (particularly kids) to see how census data was used in civic decision-making, the ABS knocked it out of the park in terms of its communication strategy and implementation.

That's a fantastic base for the ABS to build from. I think a number of people were expecting the same, or better, engagement from the ABS in 2016.

Alas, it was not to be. In 2016 little of the previous engagement brilliance is evident from the ABS.

While the ABS has repeated a level of their communication via Twitter, it's basically a shadowy repeat of their 2011 strategy - as though new management said "repeat the good stuff from five years ago, but don't update anything or take any risks".

The ABS is also remaining stalwart and largely silent in the face of several decisions which have left census collection exposed.

Their online service has been exposed as using an older and less secure security standard in order to support older browsers, rather than taking an approach which warns people and encourages them to upgrade to a more secure technology.

For non-technical people, an analogy would be the police waving past someone without headlights on a dark night onto a crowded and unlit highway in order to not slow down the traffic flow.

On another front, the ABS is confronting a surge of privacy concerns around its decision to keep names and other personal details connected to census data for at least four years. Taken without consultation with the public, this decision has raised alarm bells with privacy advocates and organisations such as Electronic Frontiers Australia, as well as with former senior officials of the ABS.

While the ABS has been fighting back to some degree, they've not really addressed the concerns in an effective way.

#Censusfail is continuing to grow as a hashtag, with a number of people considering ways to circumvent responding to the census, avoiding providing personal information or considering providing false information.

Should enough people take one of these steps it would reduce the value of the census to Australia.

I must admit that I've also become concerned about the ABS's approach, and unconvinced by the ABS's engagement on this front to-date.

I totally support and value the ABS as an organisation, and all the people that work there - however they are burning much of the goodwill they established in 2011 and potentially devaluing the census, and hurting all Australian governments through their lack of effective engagement on the issues above.

The worst thing for me is that the ABS has been a shining light in Australian government. The organisation has consistently been a leader in open data and the use of digital and social media to engage with the public.

This is important not simply for the egos of the leadership at the ABS, but is essential for good governance and effective commercial decision-making in Australia. The ABS's success serves all of us - and its failure would hurt us all.

I hope the ABS recovers from this and Australia continues to be well-served by the statistics the organisation collects.

However it would have been far better for the ABS, and all Australia, if the ABS hadn't put itself in this position of needing to recover at all.

Read full post...

Monday, August 01, 2016

Congratulations to GovHack for another fantastic year

The weekend just past featured the 6th GovHack event, involving over 2,000 participants in 280 teams across 41 locations in Australia and New Zealand working on 439 registered projects.

Effectively the world's biggest government hackathon, GovHack includes some amazing ideas on how to solve public challenges, using open data from agencies in innovative ways.

Whether you've previously heard of the GovHack event or not, visiting the Hackerspace (2016.hackerspace.govhack.org/projects), where all the registered projects are listed, is an inspiring way to start the morning and get some innovative ideas on how to address some of the pressing challenges facing your agency or organisation.

I wasn't actively involved in GovHack this year, due to family commitments, so don't have any insights from the ground on how the event went.

However from the social correspondence and general mood online, the event maintained the heights it attained in past years, while maturing further with better systems and challenge structures.

With GovHack managed by a second generation team (with the founder and key past organisers moving on or otherwise engaged), this year marked a major transition for the event.

The success of this year proves that GovHack isn't just a passion-play, but is a solid, sustainable, professional event that can become an important ongoing part of the open data movement, and tool for governments to foster citizen engagement, for a long time into the future.

Congrats to all of the organisers this year, who have made this possible.

Here's some stats from the event, based on the current information in the Hackerspace.

Total projects registered: 439
Total projects submitted: 351 (80%)

(Projects must be submitted to be eligible for judging)

The tables below show the number and percentage of submissions (Sub.) by territory, as well as submissions by 2015 population estimates.

As I measure it, the smaller the population per submission, the greater the level of engagement with GovHack within that territory - leaving ACT the most engaged, followed by South Australia, Tasmania, Queensland, New Zealand and then Western Australia, with Victoria and NSW at the end.

Projects by Country

CountryReg.Sub.% Sub.
Population
Sub./Pop.
Australia
373
291
78.0%
23,781,200
81,722
New Zealand
66
60
90.9%
4,596,700
76,612

Projects by Australian State/Territory

State/Territory
Reg.
Sub.
% Sub.
Population
Sub./Pop.
Australian Capital Territory
51
44
86.3%
390,800
8,882
New South Wales
70
45
64.3%
7,618,200
169,293
Queensland
84
70
83.3%
4,779,400
68,277
South Australia
60
49
81.7%
1,698,600
34,665
Tasmania
13
11
84.6%
516,600
46,964
Victoria
69
48
69.6%
5,938,100
123,710
West Australia
26
24
92.3%
2,591,600
107,983

Projects by Region and Local event - Australia

RegionLocal SiteReg.Sub.% Sub.
ACTCanberra
45
39
86.7%
ACTCanberra Heritage Hack
6
5
83.3%
NSW
Camperdown Games for Learning
4
4
100.0%
NSWParramatta
6
5
83.3%
NSWSydney Official
55
32
58.2%
NSWTyro Fintech Hub
5
4
80.0%
QLDBrisbane Maker Node
11
7
63.6%
QLDBrisbane Official
42
35
83.3%
QLDBrisbane Youth Node
1
1
100.0%
QLDFar North Queensland
1
1
100.0%
QLDGold Coast
6
4
66.7%
QLD
Ipswich
4
4
100.0%
QLDLogan
6
6
100.0%
QLDRockhampton
3
3
100.0%
QLDSunshine Coast
6
5
83.3%
QLDToowoomba
4
4
100.0%
SAAdelaide
36
31
86.1%
SA
Adelaide Maker
2
1
50.0%
SA
Mount Gambier
9
9
100.0%
SAOnkaparinga
5
2
40.0%
SAPlayford
7
5
71.4%
SA
Port Adelaide Enfield
1
1
100.0%
TasHobart
7
5
71.4%
TasLaunceston
6
6
100.0%
VicBallarat
9
8
88.9%
VicGeelong
5
4
80.0%
VicHack for Wyndham
5
5
100.0%
VicMelbourne
36
20
55.6%
VicMelbourne Mapspace
14
11
78.6%
WAGeraldton
3
2
66.7%
WAPerth
23
22
95.7%

Projects by Region and Local event - New Zealand

RegionLocal SiteReg.Sub.% Sub.
NZAuckland
16
15
93.8%
NZChristchurch
15
12
80%
NZDunedin
1
1
100%
NZHamilton
10
9
90%
NZNapier, Hawkes Bay
2
2
100%
NZNorthland
1
1
100%
NZQueenstown
3
3
100%
NZWellington
14
13
92.9%
NZWhanganui
4
4
100%

Read full post...

Thursday, February 25, 2016

Guest blog: Can open data really form the basis of a social good startup in Australia?

This is a guest post written by Rosie Williams, a leading Australian Open Data Developer and Citizen Journalist, who created and manages infoAus.net and writes for NoFibs.com.au. It's republished with her permission.

Can open data really form the basis of a social good startup in Australia?

I've come round to the realisation that doing open data as a business product/service (unless you are an already established business with established customer base ie not a startup) is not a realistic way to be doing open data anymore than expecting to start organisations on alternative bases such as cooperatives or charities and achieve things within a short timeframe.

After all just look how long it took for the OKFNAu to get incorporated, much less set up a sustainable financial model to fund new open knowledge projects. Now apply that level of challenge to a team of maybe one or more open knowledge advocates trying to establish similar organisations.
Trying to establish an organisation to accomplish goals whether it is under an ABN or ACN is a difficult process which is why there are so few open data projects for financial and political transparency other than my own and so few open data projects generally.

Even the OpenAustralia Foundation gets income from selling some of their services/products so the line between what is a 'startup' and any other type of organisation trying to make open data projects survive is pretty slim.

This is why so few gov hacks go on to form the basis of future projects- because of the challenges inherent in creating financial sustainability for those creating and implementing them.

If they require very little in the way of upkeep (and therefore not require full time labour) that might be plausible to continue them but most things that have value have value for the reason that they do require significant investment of labour and expertise.

When people only ever spend a weekend mocking up apps, I'm not sure that gives the true picture of the ongoing labour, expertise and lobbying required to make projects an ongoing success. For example issues in the data may not be noticeable in hack events and only come to light when more serious/ongoing engagement with the data occurs.

A better example is to compare your average hacker's daily work which requires full time commitment, a multidisciplinary team, management, contracts etc - all of which are deemed necessary in order to create and maintain successful projects. That open data projects are so often considered not to need these things to achieve the same level of stability and success is something I've always found quite interesting.

From my own experience, requiring one person to lobby to release data- wait for that release which may be months or years into the future, collect data (perhaps from multiple agency sites, clean it, design and write database backed websites, promote it etc) puts everything on the shoulders of one person or at best a very small team.

Open source projects hope to encourage volunteers to do the actual coding for free but those I know of struggle to recruit people willing to do that.  I certainly went down that road when I first launched some months ago now, but found it untenable given the short time frame put on me to prove my success.

Much open data needs tweaking in order to be made serious use of which can only take place in consultation with agencies publishing the data. It is not just a matter of agencies saying here is the data and that's that, but of a process of engagement where those working with the data can give feedback as to how the data can be improved.

This goes right into the way of making submissions to inquiries into agency annual reports or budget data sets (for example) where you are actually changing the information that eventually ends up as open data.

This sort of approach requires serious ongoing commitment and a decent level of expertise in the matters at hand.

Very few people in Australia actually run ongoing open data projects to know what it involves. Most people work day jobs doing other things (which may or may not be related to open data).

I do have a plan for going forward so that the accumulated expertise (not to mention technological output) I have created in InfoAus is not lost at such a crucial stage as the lead-up to Australia's Open Government Partnership National Action Plan.

However it is only in considering a different role to maintaining finished and regularly updated commercial grade open data projects that I have realised what a burden I was placing upon myself.
Having said that, without having done what I have over the past six months I doubt I'd be in a position to carry on in the way I intend. I'll post more about that another time.

Read full post...

Friday, January 22, 2016

How much will Australians pay for the openness and transparency we expect from our governments?

In business if no-one will pay for a product it ceases being made, or never gets off the drawing board.

Government doesn't quite work the same - many government functions and services are designed as 'public goods' - things we all need, but that many 'customers' cannot or would not pay for.

This includes services such as national defense, law and order, welfare, health care and education.

Democracy is also a service and comes at a cost - as does openness and transparency in government.

It costs money to hold elections, to release documents and data, to provide independent watchdogs that address citizen complaints, monitor agency activities and investigate corruption.

In fact measures that reduce democracy or government transparency sometime receive public support from citizens. Often this is because the consequences are not fully considered or, in a few cases, some of these individuals actually benefit from less democracy or transparency.

Many dictatorships get their start from democratic states where citizens are unwilling to invest in their own freedom and democracy. Governments on this track may gradually reduce what is visible using an economic cost argument, and foster a 'political class' that values cost-effectiveness over public good.

We've seen some of this over the last few years in Australia, with the situation of the Office of the Australian Information Commissioner being a prime example. The (much reduced) Office is now being sustained on a 3-monthly basis grudgingly by the Attorney-General, who lacks the parliament's approval to close it  down.

However it's not simply governments who aim to provide transparency into governments. There's a range of non-government organisations working in this space as well, from Transparency International to the Sunlight Foundation and Open Australia.

All of these organisations rely on funds to operate - transparency isn't free - and in Europe and North America there's well-established donors and systems for funding these groups to effectively carry out their roles.

Australia lacks these donors and systems and, it appears, even our governments are not interested in funding these independent organisations.

One such organisation is OpenAus. Founded and run by Rosie Williams out of Sydney, the service has taken a range of government data on budgets and charities and uncovered key insights that have never before been visible to the Australian public.

Rosie's work has been featured in numerous media outlets and attracted positive attention from some of the highest officers in the Australia Public Service.

However there's little in the way of funding available for this type of work in Australia. As Rosie says in her latest blog post,

There is no eco-system providing financial support to transparency projects. Projects like mine tend to veer away from government funding (to remain independent politically) and do not reflect the priorities of the venture capital ideology. As such there is a funding challenge in grassroots transparency projects in Australia that can only be filled by the citizens.

Rosie has reached out to a range of potential funding sources, but come up largely dry. Her current work has been funded through the New Enterprise Incentive Scheme (NEIS), a nine-month business building program which is set at the payment level of the dole. This is hardly sufficient to fund an individual, let alone grow a business.

Rosie will shortly finish NEIS and, having attracted only $1,500 in donations for OpenAus, is likely to have to transition back into a normal programming role.

Even if she maintains OpenAus alongside a full-time job, it will be much diminished - as will Australian government transparency.

If you think this is deplorable for Australian democracy (as I do), then please complete Rosie's survey at: https://infoaus.net/openaus/survey.php

You may also wish to donate via the OpenAus site: https://openaus.net.au/contribute.php

Remember that if Australians are not willing to pay for the openness and transparency we expect from our governments, then we will get what we deserve - a much diminished democracy and more opaque state.

Certainly we should expect governments to use more of the funds they already collect from us to support transparency - 'reporting back' to their 'shareholders', citizens.

However for truly independent views on our government citizens need to directly contribute - via effort or funds - to organisations such as Rosie's.

Contribute now at: https://openaus.net.au/contribute.php

Read full post...

Monday, January 11, 2016

DataStart announces eight shortlisted open data startups

Late last week the shortlisted start-ups for the DataStart program were released - here's why it's significant and what happens next.

In November 2015 the Department of Prime Minister and Cabinet, in parternship with Pollenizer Ventures, announced Australia's first open data commercialisation competition, DataStart.

Designed as a pilot to test the approach, entrepreneurs, data scientists and open data enthusiasms were invited to apply for a program that would see up to 20 founders shortlisted, trained and one winner receive start-up coaching and potentially up to $200,000 in funding (via Right Click Capital) towards becoming a commercially viable company.



The program attracted mixed reviews. While some applauded the efforts to link open data competitions with actual commercially viable ongoing outcomes (which has been an ongoing criticism of data competitions in Australia), others saw it as a 'winner takes all' process with little value to the community.

My view was in-between - we need programs like this to be piloted, with the best becoming part of the startup and open data ecosystem. However we also need governments to fund their open data programs such that datasets are released at a sufficient quality level and reliable frequency to be a commercialisable resource.

The DataStart program attracted over 200 entrants and late last Friday Pollenizer released the eight shortlisted start-ups, consisting of 20 founders.

These founders begin a five-day program this week in Sydney to test and work-up their start-ups to evaluate whether there's truly a commercial basis for the ideas.

Following this, based on the competition guidelines, a single start-up will be selected to go into a 9-month incubation process at Pollenizer in Sydney, with the potential to also secure $200,000 in funding from Right Click Capital on commercial terms (aka in return for equity or other consideration).

It's great to see the level of interest in this program, and the next step begin.

What would be really good to see is a higher level of transparency around the start-ups and founders, featured interviews, examples of what data they are using and how.

This is the challenge in public-partner arrangements, where often the partners have a different set of values and expectations, as well as different obligations under law and policy.

I'm hopeful that efforts are underway to align these expectations and values and ensure that these startups become role models and examples of how open data can be used commercially, rather than get hidden away under commercial-in-confidence arrangements.

Of course their IP needs protection, but there's a lot that could be promoted without breaching commercial confidentially.

Who are these founders and start-ups, why are they using open data, what problems are they solving?

Hopefully we'll learn more than their names and vague details of their project area over coming weeks and months.



Read full post...

Friday, January 08, 2016

Contribute now to Australia's Open Government Partnership National Action Plan

Now that Australians are heading out of holiday mode, it's a good time for a reminder about the Australian Government's process to become a member of the global Open Government Partnership (OGP), through developing our first OGP National Action Plan.

If you've not heard of the Open Government Partnership, in a nutshell it's an international group of governments and civil society organisations committed to progressing open and transparent governance in the 69 participating nations through a cooperative and supportive process.

Each nation makes a commitment to improve their national government's openness and transparency through a statement by their government, supported this through a two-yearly National Action Plan (NAP) with priority activities relevant to the nation's development stage. These NAPs must be developed cooperatively between government and civil society through an active consultation process.

Nations are assessed annually on their NAP progress through an independent process, with their achievements and shortfalls highlighted internationally.

Australia was invited to become a founding member of the OGP back in 2011, however deferred this decision until 2013, and then delayed it further after a change in government.

The current Prime Minister re-committed to the process late in 2015 and restarted the process to become an OGP member.

I ran a series of Information Sessions across Australia's east coast about this process, funded by the Department of the Prime Minister and Cabinet, and the presentation used through these sessions is below.



Currently Australia is consulting on the commitments that should be prioritised in our first National Action Plan, with Australians invited to provide ideas through the wiki (ogpau.wikispaces.com) or via email to ogp@pmc.gov.au

So if you'd like to see improvements in how open and transparent federal government is in Australia, please contribute via the channels above.

This stage of the consultation is open until the end of February, so don't wait too long!


Read full post...

Thursday, December 03, 2015

New data report signals a major shift in data thinking in the senior Australian public service

While successive Australian governments have touted 'evidence-based policy', a significant ongoing challenge within Australia's public service has been generating or accessing the right data at the right time - particularly across agencies - for actually testing policy assumptions and following a more agile policy development process.

I've personally witnessed the challenges within agencies of being able to locate useful and usable data for program design and delivery assessment, and the mechanics for sharing data between agencies, requiring formal agreements and long discovery and lead-times.

One of the more signficant side-effects of the open data revolution has been to highlight to agencies how little they know about the data they already collect and hold, who owns it (agency or external parties) and how hard it can be to share productively between agencies, let alone with the public.

A significant initiative this year was the commissioning of an internal data report by the Secretary of Prime Minister and Cabinet, named the Public Sector Data Management report.

This report was designed to help the Australian Public Service (APS) understand its internal data landscape and recommend steps that could be taken to improve how data is generated, managed, discovered, shared and therefore used in policy and program development and evaluation.

Put simply, if the APS can get its approach on data right, suddenly it would become far clearly to both internal and external stakeholders how effective various government programs were, and clarify policy gaps and opportunities.

This internal report, now released publicly under Creative Commons licensing, offers a glimpse into how effectively data has been used in the APS, and recommends a strong series of measures to strengthen data collection, sharing and use.

If anything speaks to the commitment within the APS to use and share data more effectively, this report does. It's worth not only a read, but a close study.

The Public Sector Data Management report is now available online via the data.gov.au blog at blog.data.gov.au/news-media/blog/public-sector-data-management.



Read full post...

Bookmark and Share