We are a small, flexible team working for the UK Parliament, with a passion for building user-friendly & accessible digital applications.

http://myparliament.info

MyParliament

Our latest piece of work is a new website, http://myparliament.info, aimed at engaging members of the general public in the work of Parliament, so that they can easily find out about key business that has taken place and can follow the activities of individual Members.

As well as being able to browse debates, votes and review historical composition of the Houses of Parliament, you can type a word or phrase into our search engine and find Members by name, location and, for MPs, constituency. And if that wasn’t enough, you can also search the text of speeches and oral questions, the text of written questions and answers as well as the text of Early Day Motions. All your results are shown in one place and you can explore more detail from there.

We’ve also built a set of profile pages for Members, where data from a number of sources is collated in order to provide a view centred on the individual Member.

The site is now in its first public beta release and we would very much like to hear your feedback about what content we’ve made available so far, its presentation (we’re particularly proud of Hansard), and what you might like to see in the future – we would love to add some more ideas to our list below.

Some of the work that will be released over the next few weeks includes…

  • A richer search engine. e.g.

Show me all the Lords with an interest in education

Who is the Home Secretary?

What EDMs have been tabled this week?

What questions did a particular MP ask last month?

Show me last weeks PMQs

  • Including financial interests of Commons and Lords members
  • Information about upcoming Parliamentary debates and events
  • Much more!

Most importantly of all, the site is designed to be used on any device from a smartphone to a tablet to a desktop and is simple to read and use. Please feel free to share this as widely as possible with anyone you feel would be interested.

I hope you find it a useful tool and please do follow us @digiminster or keep an eye on our blog for future work on this and other applications going forward.

Interactive Research Paper – General Elections

Ahead of the 2015 General Election, we have been working with colleagues in the statistics section of the House of Commons Library to produce a new site that can be used to display detailed information about election results to a wide audience. We present General Elections Online – or GEO for short.

The statistics team already produce a research paper after every election, however this incorporates a very detailed analysis and therefore takes time to produce. It is also only created as a pdf which means it isn’t as digital as we would like it to be, so this site is really the first fully interactive research paper produced by the Commons Library!

Right now it contains data up to and including the 2010 election. You can review a map of the UK with constituencies colour coded by party, see results at constituency, county, regional and country level with historical data going back to 1992. It is also possible to review data for each party that won a seat, and delve into a big bucket of statistics. What is the 371st safest seat in the Commons*? You can find out on the statistics section of our site…

It’s our first version, there is still work to be done – we want to allow searching by postcode and constituency name, as well as making it look a little better on smaller devices. Right now its responsive down to a tablet size (landscape is better), but we will make it more phone friendly soon.

And then we’ll be testing with mock 2015 data, so that as soon as possible after results are declared next May, the site will allow you to see what has happened on the electoral map. More detail will, of course, still be available via the next research paper for those keen on in-depth analysis.

*It’s York Central BTW.

One year on

We’ve had a bit of a break from blogging over the summer recess,  but as the team completed sprint #26 just over a week ago, and so was officially one year old, it seems a good way to restart the blog with a little celebration of our achievements over the past twelve months.

Through direct engagement with our customers across the parliamentary estate for the majority of our work, plus some excellent feedback from external users for the work we’ve been doing on the Members Data Portal, we’ve produced a number of excellent systems and apps over the first year of our existence. Some of the highlights that we’ve not written about before (such as Lords Divisions and Solution Creator) include…

Dev Dashboard

Dashboard

A tool for monitoring builds, servers and other info key to the good running of a development team. With personalised audio whenever a build breaks, fixes are never far away!

 

Catering

Admin tool for catering teams to manage menus across Parliamentary catering outlets. Data service allows quick reuse across the intranet of data to save double keying.

Admin tool for catering teams to manage menus across Parliamentary catering outlets. Data service allows quick reuse across the intranet of data to save double keying.

 

Lords Digest

digest

Tablet friendly, digital presentation of the House of Lords Library Current Affairs Digest. The first Parliamentary publication to be fully produced as HTML5 content by the content creators, a superb example of digitising an existing process by developers and business staff collaborating.

 

Digital Library

Digital Library

Building on the success of the digest, the Digital Library aims to offer an enhanced vote bundle for both Houses in electronic form. Initially by exposing existing pdf data, and iterating over time to move to a richer HTML5 based experience. This screen shows the admin tool, a blog post about the app itself will follow soon!

 

Calendar

calendar

A revamped and updated Parliamentary calendar maintenance tool is due to go live very soon. This will speed up data entry and allow information office staff to enter (and publish to the public) a great deal more information about events taking place in Parliament.

 

Order Paper

orderpaper

The core part of the e-procedural project for the House of Commons is the new Order Paper application. Turning a fully paper based system into an electronic one that allows full reuse of data with a number of other systems is a longer term project but one that has nonetheless had successful trials over the past couple of months and we hope to see an initial iteration move to live late this year.

*All data in the above screenshots is for demonstration purposes only

All in all we’ve built or extended over 20 projects over the past year. Not a bad achievement, and a great foundation for the future.

Will be updating our Systems and Apps pages with these summaries soon.

What next?

Watch this space!

Extending Member data and more

Over the last sprint I was fortunate to be interviewed by Harvey Nash as part of their Technologists in the Spotlight series, which was great. The team also said goodbye to @MartinSpain, our excellent UI developer who has moved on to a new challenge.

All this aside we’ve been working on a few things that we’ll blog about in the next week or so, including using IP enabled sensors to track office temperature and monitoring if we’ve been sat still in our chairs (creating great software…) for too long so we are sure to take a break and stretch our legs. Really pleasant change from more traditional development and I’m sure you’ll enjoy reading about it soon.

Our work on digitising production of the Commons Order Paper continues apace with a business trial of the first phase of work taking place before summer recess. It’s been the first large scale project the team has worked on and we are looking forward to hearing feedback from the users – a number of blog posts on how we’ve built this system will appear over the summer.

Our Members Data Portal goes from strength to strength. This sprint we’ve added the ability to search debates and divisions by title for when you don’t know exactly when something happened, or perhaps when you want to know how often a debate on a topic takes place (finance debates happen a lot!). Debates are searchable by contributing member and divisions by majority. You can access this via a new tab control on the calendar pickers on both the debate and the division data screens.

Division results now include some simple visualisations to show the split of the vote by party (correct colours to follow!), and we’ve also included some charts on the historical membership pages to show house makeup on any given date.

Debate feeds are now easier to browse through thanks to a simple forward/back navigator tool, and we’ve added a list at the start of each debate of all the contributors.

Watch Video

But perhaps most excitingly of all, we now link debate content to the accompanying video feed hosted by ParliamentLive.tv. Hansard record timestamps throughout the day, including at every division, and we’ve been able to hook our feed directly to the timecode in the video feed for the debate. Right now this will only work on laptops/PCs as the video stream is served via Silverlight, but much work is going on to improve this by the folk in the AV team and so watching archive content via a mobile device isn’t too far away. I should also point out video is not available until the following business day as the archives are not updated until the house rises but you can always watch a live feed on the ParliamentLive website directly.

For me it has been really interesting to see how tough the job of a Hansard reporter must be when watching the video feed and then comparing it to the transcription. It’s pretty cool how much AV data exists as well, I hope you find reviewing some of the debates that have happened in Parliament over the past year as fascinating as I have.

Reading chamber debates

MyParliament now includes debates for the Commons and the Lords!

A sample debate

 

Following on from our work of the last sprint, this past fortnight we’ve been working on exposing the transcripts of debates from the data we’ve captured. There are currently on the parliament website a couple of ways to review Hansard. The rolling page, which exists whenever the house is sitting, and then the historical pages where you can review content from pretty much any date in the past.

Our sprint goal was look at ways at creating functionality that extends beyond what is currently on parliament’s website, to build a common look and feel for the content regardless of the date, and to enhance the work we have already done to make divisions and contributions searchable by linking directly from search results to the debate where the result happened. We also wanted to experiment with the UI and see if we could bring something a little closer to a “chat” type interface to a debate transcript.

You can now browse debates from May 2013 for the Commons and the Lords using the calendar picker (same one as we’ve used elsewhere on the site), days with content will be highlighted for ease of use. Once a day has been chosen, a list of all the debates that happened on that date will be shown and from there it is simply a case of clicking to have a read. When reading a debate we have permalinks for every contribution, and when searching for divisions or contributions via options from the menu, results will link you straight to the pertinent section of the debate.

If all this wasn’t enough, we’ve made some tweaks to our previous work as well – you can now search freely for both divisions and contributions with no restrictions on date ranges. To enable this we have built pagination onto the search result pages and enhanced our API to also support skip and take on the results. We’ve improved our date picker to make it easier for you to navigate the site too.

Our next sprint, which will likely be the last one on this for a little while, will bring some better searching of debates and divisions, some data visualisations and hopefully a little bit of fun with video…

Member voting and contributions

Over the past few sprints we have been looking at ways we can show some more information about members, using data we already have and by adding some value to the content produced by the Hansard team to expose the huge amount of information they produce in some different formats.

So we’ve come up with the Members Data Portal. At the moment this is somewhere between an alpha and a beta, but we want to share it now as if we keep waiting for something we are totally happy with we’ll likely end up never deploying. The data is not official or 100% accurate (yet) but its a good start point I hope you’ll agree.

The first set of information we are exposing is the memberships of both the Commons and the Lords. This allows you to review how the make-up of parliament has changed over time. This data is sourced from the Members Data Platform and makes interesting reading in terms of not only majorities, but also how the gender balance has changed over time as well.

Right now there is no parliamentary data service that offers easily searchable results of divisions in the Commons (but we do have for the Lords and it’s getting better), and as our MPs are representing all of us, it seemed as good a place as any to try to expose in an easily searchable format how MPs have been voting.

We have created a couple of different takes on this data. Firstly you can review divisions for any given day. At this time only divisions that took place in the chamber are available to search (deferred divisions are recorded differently and we haven’t tackled that just yet). On the calendar control you will see certain days shaded. These are days with divisions recorded. Click through to review what divisions happened, what they were about (we link to bill information where we’ve been able to work it out!), who voted and how.

A second take on this data is how a specific member has voted, this tool allows you to choose a member (find yours using our MP finder tool) and review how they have voted in divisions over a set time period. Hit the all divisions checkbox to see the divisions that an MP did not vote in, now there are any number of reasons this could have happened, but it is interesting to see when it did.

Both of the above are also available for the House of Lords, although this does exist elsewhere we want to do more with debate outputs in future so this seemed a useful thing to do in passing.

Finally (and maybe most excitingly) we have created a tool to enable full text searching of contributions for both the Commons and the Lords, essentially who said what and when. Simply choose a member, or maybe search all members, enter a search term and away you go. This makes it very easy to find out who is talking about the key issues of the day. Results are ranked by relevancy to help you find what you are looking for.

We have captured data from the start of the last session (8th May 2013), that is 90,000 contributions at the time of writing. Due to the size of the data set, right now we are restricting all queries to a maximum of 3 months of data at a time, but will soon add paging to the results enabling you to scour all the data in one go.

Coming soon… outputting the content of debates in a more modern way and linking to them from divisions and contributions to help give more context to what has been happening, we want to do some data visualisations around what we’ve captured, and we will expose the underlying data service that powers our portal to anyone who is interested.

UPDATE (3/6) : The data service is now available here

Offline web applications with HTML5

The Rapid Apps team builds its apps using HTML5 as we believe this is the quickest way for us to create and release apps that can be used on any device without the need for lots of bespoking for each of the platforms that are out there. One of the key things needed with an app is the ability for it to work offline using  a manifest. Here’s how we do that.

When working with an HTML5 web app that you want to run offline you will need to rebuild the manifest (over and over and over) whenever you make changes to the code so that your browser knows it has changed and you can test/use it. This is a pain, so we have created this nice little console app that you can run in the background and it will keep the file up to date for you.

Download the manifest generator

How do I make a web app cachable I hear you ask?

To make a web app cachable, you first need to tweak the markup.

In the <html> element you need to add a manifest attribute that points to your appcache file.

e.g.

<html manifest=”fymp.appcache” xmlns=”http://www.w3.org/1999/xhtml”>

The appcache file needs to be served by IIS as type text/cache-manifest, so you’ll need to tweak your web.config file. In system.webServer add a staticContent element and then a mimemap for the file extension. ALWAYS remember to put a remove element in first incase the server hosting your app is already configured to serve appcache files, otherwise it will fail miserably as you can’t have the same mimeMap twice.

e.g.

<system.webServer>
    <staticContent>  
      <remove fileExtension=”.appcache”/>
      <mimeMap fileExtension=”.appcache” mimeType=”text/cache-manifest” />
    </staticContent>
</system.webServer>

The appcache file should be called {application name}.appcache and live in the root folder, where the index.html page is kept. In addition to this, you also need to create a file called AppCacheExclude.txt which lists out the files/folders you don’t want to put in the cache (e.g. any visual studio files, web.configs and the like). An example of the contents of an AppCacheExclude.txt file is shown below.

AppCacheExclude.txt
mycache.appcache
MyApp.csproj
MyApp.csproj.user
MyApp.sln
MyApp.v12.suo
Web.config
bin\*
obj\*
Properties\*
.svn\*

Using a * means all files/folders in the folder specified. You must include this or it will fall over and die on you horribly. It is also, obviously, crucial you don’t cache the exclude or the appcache file.

With all the above done, simply run the console app, tell it where to monitor and off you go. When you are done coding the manifest file can be checked back in to source control so CI tools can pick them up, deploy them, and then all the consumers of your app will seamlessly receive the changes!

If you are really interested (and I know you are) you can get the source code here

Speeding up new projects

We wouldn’t be the Parliamentary Rapid Apps Team if speed wasn’t important to us, and to our customers. As you can imagine, we are constantly picking up new projects and aiming to get a working product to our clients as quickly as we can, and one of the best ways to speed up that process is to speed up the time it takes to kick the project off.

Whenever we start a new project, someone has to go through the same steps just to get to a point where we can start coding for real:

  1. Create a new code repository for the project
  2. Create the shell of a project in Visual Studio
  3. Import useful features and 3rd party tools into the project
  4. Create TeamCity projects and builds for the project so it can have continuous integration and build monitoring, and be one-click deployed to the QA environment

Each time we do this, there is also potential for steps to be missed, copy and paste errors to occur or naming conventions to be ignored. And just as importantly, it takes time!

So simple a computer could do it

The solution to this problem is to automate this process as much as possible to cut down on manual mistakes, save time and make the most of our reusable code. So we have created MEGABuild™, which consists of two main components.

Templates

The first step was to identify which type of projects we create the most often (.NET MVC and .NET WebAPI) and what components and code we used in those projects repeatedly. We then created a Visual Studio solution for each type of project, which can be used as the perfect starting point for any new project. Our MVC solution template, for example, includes JQuery, Bootstrap, log4net, our custom stylesheets, a simple view layout and our basic configuration and automated build settings. Each solution is built with a number of keywords in it that can be replaced automatically when we use it to create a new project.

MEGABuild™

The final piece of the puzzle is a simple-to-use wizard that allows us to enter key information about a new project (such as its name and what namespace we should use) and then works away for a few minutes doing all the steps we would normally have to do manually. As a brief overview, it does the following:

  1. Checks out the latest version of the Solution template
  2. Creates the SVN repository for the new project
  3. Copies the solution template into the repo, and renames the keywords with the new project settings using powershell scripts
  4. Updates the ignore list for the new repo
  5. Commits the new solution to the repo
  6. Creates a new project and subproject in TeamCity
  7. Creates automated builds in TeamCity based on the build templates we have previously defined
  8. Finally, shows the user where they can find all this new stuff!

 

When the wizard has finished, you can simply checkout your new project from the repository and start coding!

Even more mega

We hope in the next couple of sprints to make MEGABuild™ to do even more for us. It can automatically enter DNS entries for our internal environments, and add details about the projects it creates to our other tools, such as our build monitor and our release management tool.

Digitising divisions

How the clerks will record divisions in the Lords very soon.

How the clerks will record divisions in the Lords very soon.

As mentioned in my last post, we have been working on a new system for the Lords to record their divisions electronically, rather than via the current method of marking off against a printed list that’s been around in one form or another for, well, ever. I would like to share a quick overview of the development process that went into building this new system.

The project came along a little while back when we had a chat with the business development manager for the Lords who is very keen to look at ways of improving how the business of the house is done. He came to us with some simple wireframes that he had designed, asking us how easy would it be to create a system that would allow touch based capturing of a division which could plug into existing systems post capture so as to keep the rest of the workflow ticking along as it does now. We said “lets create you some prototypes!”.

So we did. We built out over the course of a two week sprint three simple HTML pages that presented the member list in different ways and gave it to our customer to try out with a group of stakeholders and let us know which of the options they would like to take forward. We did this because although it was fantastic to have the initial wireframe to start things off, it is often quite different when you come to the realities of using software in the real world and so it was a safer bet to create some quick “throwaway” prototypes than to build out a big old system that turned out to not be what people wanted.

Once the prototypes had been tried out and the decision of which one to use had been taken, we cracked on and turned it into a fully working system. Utilising data from the awesome Members Data Platform to generate the lists, we built the system for the division clerks using HTML5, CSS3 and trusty JavaScript with the back end services that receive, collate and push data into the existing systems built out using MVC5 and WebAPI.

Showcases were carried out every sprint, usually to just a couple of stakeholders, but as we finished the beta version of the system we ran a demo to all the clerks so they could all see what was coming their way. To make things a bit interactive we ran the demo as if a division were happening and they were Lords (each got a name badge) and the dev team were the clerks. The question on the table was “That marmite is a most tasty spread”. Sadly the motion was defeated, nevertheless it was a great way to show the users how things would work in the real world and we had a good Q&A session afterwards which immediately led to a few tweaks in the next sprint.

The system is now in UAT, has been well received, touch devices have been purchased for the division clerks to use to record the votes (the 17″ beast tablet was sadly not feasible but it was an awesome bit of kit!) and as I said earlier we hope to go live over the summer.

This sprint we have mostly been…

Over the last couple of weeks the team have been working primarily on two projects for colleagues in the House of Lords.

The first of these is a new system for the recording of divisions. Currently a paper based system is used by division clerks to record how members vote when a division is called. We have been asked to create a new electronic system that will allow the clerks to record votes on a touch screen device and submit their results to a central location (the table office for those familiar with parliamentary process). The clerks in the table office will then verify the division, add some metadata and submit the results into our internal systems, including the soon to be launched data.parliament platform so that records of how lords are voting, and what they are voting on will be available to any interested party as quickly as is possible. Plus it will save a ton of paper! We’ve been working on this system for a few sprints now and are hoping we will see it go live over the summer.

Next up we have begun development on a similar concept, the recording of attendance for members of the Lords. Peers get an allowance based on the days on which they are sitting in the house so ensuring this information can quickly and easily be captured is very important. Thanks to the work completed on the divisions project this piece of development is much easier than it otherwise would have been and we will be showing the first iteration to our customers at our fortnightly showcase tomorrow. Donuts are hoped for…