How much CPP have I earned so far?

Estimating your CPP pension has always been something of a dark art.  Service Canada has an estimator but it's only an estimator and only works for the common case.  If you spent significant amounts of time outside of Canada or have a spotty work history then the estimator will be wrong.  So I put together a spreadsheet which will let you compute how much you have earned so far.

This will produce how much pension you would be entitled to if you stopped working and waited until 65 to collect CPP.  To fill out the spreadsheet, you will need your statement of contributions from Service Canada or your annual earnings from Revenue Canada.  You can estimate your salary for a reasonably close result.

Fill in the Actual Earnings column with your gross pay OR  fill in the Pensionable Earning column with your statement of contributions from Service Canada.

Fill in your Data

The final number at the end is the pension you have earned so far in present day dollars.  It doesn't impute future earnings and CPP will adjust the amount upwards for inflation.

What do you mean I'll only get a pension of $150!?

That's only the pension you have earned to date.  So have a guaranteed pension of $150, your pension will grow as you work more years in Canada.

I'd like to thank Doug Runchey for How to calculate your CPP retirement pension for providing all the math used to calculate the CPP payments.

Why Government software projects go wrong - Part 2

Last time I mentioned the problems that are caused by governments in government projects.  What about the other side - the ISVs.  It turns out that they aren't entirely blameless.

Bean-counter Effect

As I said previously, government work is very risky for small ISVs so much of the space is dominated by large ISVs.  Large ISVs exist in a space where they are controlled by the bean counters.  I really want to emphasize that certain changes in corporate culture are necessary as the company grows but this one is very detrimental.

The bean-counter effect is treating employees like interchangeable cogs in terms of service delivery.  Two software engineers with 5 years of experience are completely equal as far as they care, even though that is provably false.  To them, a team of 2 top-notch performers earning $100K/year is worse than a team of 3 middling performers earning $60K/year.  Switching to the latter means saving $20K/year in payroll and an increased their ability to bill out clients.  After all, the clients still pay the same rate for the best and the worst.  In the end, the ISV has a perverse incentive to employ low-skill, but highly profitable, developers which end up dragging out the project.  Needless to say this is very bad for the client.

Sales over Quality

Many ISVs will offer to "customize" their current solution to meet the client's needs.  Here's the dirty little secret of programming.  All software platforms represent a trade-off, by optimizing to do things well, like geo-spatial analysis, they will do other things much worse, like community interaction.  The only way to counter-act this is to be equally good at doing everything, however, this ends up with a platform which is equally bad at everything.

In the best case, the existing solution is close to what the client needs.  In this case, the client is well served by re-using an existing solution.  However, in many cases Sales will try to make a purse out of a pig's ear (after all, it's their job).  In cases such as turning a geospatial platform into a social networking system, it ends up being worse than starting over from scratch.  However, the perverse incentive is that many clients won't, or can't, pay for a proper custom solutions.  So, "customizing" an existing system, no matter how much code has to be written or discarded to make it fit, is better than starting fresh with a targeted solution.

Fear of Change

"We can't develop a new version because it would offend our current clients."

Many agencies are so paralyzed with fear of change that they refuse to analyze new technology.  Part of being an ISV is reviewing all the new technologies that come out and determine if they meet the needs of their clients.  After all, clients generally won't have the needed expertise in-house, it's why they hire ISVs.

What this means is that most of these organizations will adopt very stale technologies and use them as the basis for new projects.  This can be caused either by the ISV deciding its investment in the technology is too large to change or by the fear that clients won't be interested in a new system.  What this means, is that clients end up with systems which are expensive to support and nearing a very sharp end of life.

In addition to the old technologies, these large ISVs will maintain old software development techniques and practices.  Waterfall, big-design upfront and The Methodology is often used to attempt to produce a quality result.  By staying behind the curve on modern software development practices these ISVs think they're being consistent but all they're doing is harming the client.


In the end, I don't think anyone at the ISV is deliberately trying to do a bad job.  They are doing their best, but I believe their incentives don't line up to produce amazing software.  Rather, they're incentivized to produce software which has the right superficial features, of which sadly, "Great" is not on the list.

Why Netflix autoplay is broken by design

Netflix has a cool feature where after you are finished watching one episode of a TV show, it will forward you on to the next one.  It's a neat idea but it's implemented so badly as to be border-line user hostile.

It starts before the previous show ends

 

When the show your watching gets to the credits, it is crunched into a mini-box where in the credits play out in miniature.  This is the same thing that happens with broadcast television except that Netflix isn't showing ads for anything except for other shows you might want to watch.

Hey, I was watching that.

Some people view this as disrespectful to the creators of the show, which is an interesting argument but not one we're going to bring up here.  A much better argument is that many shows don't just show credits at the end, they add in an extra scenes or follow-ups.  This is most famous from Arrested Development where the producers added a false 'Next Time on Arrested Development'.  So the system is actually removing value from me.

It fails at actually auto-playing

 

Netflix will actually start playing the next episode of the show you were watching.  Which is nice, if that's what you want.  The trouble is that's implemented so badly that no one would possibly want it to work that way.  What it does is play for a few minutes into the show and then stops playback to ask if you want to actually continue watching the show.

Seriously, I was watching that.

So now, I have stop what I'm doing and go instruct Netflix to actually auto-play.  This is UX equivalent of stepping on a Lego.  It's just bad and needs to go away.

You can't turn it off 

 

The final sin is that I can't configure it one way or the other.  Maybe you like it this way and that's fine, but what I want is the ability to configure it operate differently.  I understand that more config options will increase testing, but there has to be a reasonable middle ground.

A simple option to turn off 'auto-play' would have hugely improved usability.  You can leave this feature on by default, but I know would turn it off given the choice.

Developing with GeoServer

If you're developing an application which uses GeoServer to provide mapping data then you might end up in a situation where you need to run GeoServer on the same Tomcat instance as your webapp.  Usually this is to do request proxying to the bigger GeoServer host.

In order to set Tomcat up to properly deploy GeoServer you need to make a few modifications if you're running it in Eclipse.

In the Overview page of Tomcat server set the Server Locations to 'Use Tomcat installation'.

Next, extract the geoserver.war file to /tomcat/webapps/geoserver and configure the Tomcat server.xml with the following statement.

            <Host appBase="webapps" autoDeploy="true" name="localhost"
                unpackWARs="true">

  
                <Valve className="org.apache.catalina.valves.AccessLogValve"
                    directory="logs" pattern="%h %l %u %t &quot;%r&quot; %s %b" prefix="localhost_access_log."
                    resolveHosts="false" suffix=".txt" />



                <Context docBase="E:\tomcat\webapps\manager" path="/manager"
                    privileged="true">
                </Context>

                <Context docBase="E:\tomcat\webapps\geoserver" path="/geoserver">
                </Context>

            </Host>

Finally, edit Tomcat start up arguments and add "-XX:MaxPermSize=128m" to the VM arguments.

Why Government software projects go wrong - Part 1

If you have been following the news then you might have heard about British Columbia's new Integrated Case Management System which is a complete disaster.  It turns out that the $180-million dollar system is completely unusable due to bugs and is exposing extremely sensitive data to unauthorized users.

So how did it get this way?  How did $180 million disappear down a hole with only a horribly broken system coming out the other side?  While I was not involved in this particular project I think I have a few ideas of what went wrong.

I think there are two culpable parties here, the government and the contractors.  Let's examine what the government side is doing wrong.

Poor Focus

If you've ever dealt with gathering requirements the first thing that happens is absolutely everyone is consulted on what they want the system to do.  So what happens is that a kitchen sink-full of requirements is drawn up.  Every single person from the politician selling it to his constituents, to the higher level directors, to the lady with too many cats in HR is asked to throw in their advice.  Funnily enough, end users are rarely consulted until the finished product arrives.

Now it's not bad that extensive lists of requirements are being drawn up but sadly that's where things end.  There is no prioritization or optimization.  So absolutely crucial requirements like 'must protect vulnerable data' and 'must be customizable to official ministry colour schemes' are of equal weight.  Countering this are the excessively detailed requirements like must use a 1024x768 display or must use technology 'X'.  Is it impossible to know if these requirements are truly firm or if they exist solely as a reaction to getting burned in the past.

This is the first major pitfall, by making all requirements equally important it is incredibly difficult to properly discern what truly matters and what can be skipped.  Where does the client derive value?  What are their major risk factors?  The end result is a hugely expensive, incredibly complicated system that doesn't properly address real concerns but does cover-off buzzwords in the contract spec.

Lack of planning

A big problem is the lack of planning.  Not at the department level but at a higher up level.  Often a project is stalled or delayed endlessly while waiting for a budget to appear.  So a competent provider is found and a plan of work is created and then... nothing.  The project languishes for months while waiting for a budget to appear.  This is a major problem for smaller ISVs who don't have a diversified revenue stream and can't wait around.

The US government has not passed a budget since 2009.  3 Years.  Agencies are limping along on continuing resolution and are unable to make any long term plans.  These agencies can't make long-term (ie, more than 4 months) commitments because they might be shutdown without warning. 

Finally, most government agencies think the clock to begin development starts when they talk to the vendor not when the contract is signed.  So ISVs are expected to bear the cost of speculative development in the hope that the agency will follow through.  Needless to say this drives up the risk to almost unbearable levels for most small ISVs and drives all the business to large ISVs.

No Oversight

Perhaps the most important thing lacking is proper oversight.  In the age of outsourcing governments are pushing larger and larger sections of work out to private business.  While this is fine, enough talent must be retained internally to be able to competently evaluate and monitor the progress of the project.  This person needs to be aware of the correct ways to keep tabs on software projects and determine actual progress of the development.

Software development is usually quite different from other development projects and very few people outside of the industry have the proper knowledge.  Common tools like Gantt charts and percentage done numbers are inappropriate measures but are expected to be used.  Without oversight it is very easy for the ISV to extract unreasonable amounts of money from the government by dragging out contracts.  Without a strong focus on the end goals projects can wander which is good for the ISV but not the client.

Even if the ISV isn't interested in milking the contract, internal forces can cause the project to wander.  Shifting political goals and management by committee prevent solid objectives from forming.  With shifting objectives it becomes incredibly easy for vast sums of money to be 'wasted' when goals are changed.  Pretty soon the project is late, over-budget, and of poor quality.


In the end, governments reap what they sow when they play games with new software development.  However, it would be wrong to lay the blame exclusively at the feet of government.  Next time I will discuss how the ISVs screw up projects on their end.

American Netflix in Canada

American Netflix in Canada

Have you wanted to watch American Netflix elsewhere in the world but don't want to pay for a US proxy?  Turns out there is an easy way to do thanks to IPv6 and Hurricane Electric.

The process is really simple, sign up at http://www.tunnelbroker.net/ and create a regular tunnel.  Select the nearest end-point to you to get the best performance.  From there, follow HE's example configuration to get your IPv6 tunnel set up on your PC.

The downside to this is you will no longer be able to view your regional films in NetFlix until you disable the tunnel.

What's going on?

It turns out that web browsers prefer to use IPv6 over IPv4 when available.  By configuring an IPv6 tunnel you are being assigned an address from the upstream provider's pool.  If your provider is American then you will appear to be American.

 So when you connect to netflix.com you will come in over a US-provided IPv6 address.  Netflix will then attempt to geocode your address and it will determine that you are, in-fact, in the US.  This is the fundamental flaw in using IP address to determine where a person is located.  IPs do not correspond to a person's physical location.  At best, IPs can reliably tell you where the upstream provider is registered, but if they're providing service to people in Tonga and Britain you won't be able to tell them apart.

Introduction

There are many blogs on the Internet which discuss software engineering in very technical terms.  While I have found these incredibly useful professionally I don't think they really help people who are not completely steeped in the culture. 


I have found that many people seem to under-estimate the need for clear communication; at least in themselves.  What I am trying to cover is software engineering in a clear and concise manner.


Understanding is my goal.