Unauthorized Access to Network

Summary

A bug in the Cloudtrax voucher system we use to manage teacher access to TeacherNet enabled the voucher code "123" to be used to connect to the network. Students at the school discovered this, and used the code to connect to the network and use its wifi.

Other than students being able to access the Internet from their own devices, there was no other impact: no other systems were affected, no teacher access was affected, no other voucher codes were affected.

We were able to identify the source of the issue quickly, and put systems in place to prevent continued use of the voucher code.

Detailed Description of Events

On Monday, May 13, 2013 we received word from a teacher at the school:

So I am not sure what is going on but, as of today, my TeacherNet password will not work at all BUT the password 123 will work. I watched [name redacted] use it today and get on when I couldn't, from the same laptop! 

We checked the following in Cloudtrax:

  1. Confirmed that the teacher in question had an expired voucher, which explained the "my TeacherNet password will not work at all" issue.
  2. Confirmed that there was no voucher code 123 (there was a voucher code containing 123 which we disabled, just to be sure).
  3. Found the setting of "Vouchers work on all networks" was on (we had deliberately set it to that value thinking that "voucher portability" was a good idea); although there was no other voucher on any other network under the TeacherNet master account with voucher 123, we unchecked this option just to be sure.
  4. Under the Advanced Settings tab we found the setting of "Disable Automatic Upgrades" was on, which meant that the nodes on the network would not receive any automatic firmware upgrades; if the issue is old firmware with a loophole in it allowing the 123 voucher, this setting would prevent an automatic upgrade, so we unchecked the setting. Before unchecking this the firmware versions on the nodes were:

    PRINC-347-07
    fw-ng-r376
    batman-adv

    PRINC-347-08
    fw-ng-r397
    batman-adv

    PRINC-347-04
    fw-ng-r376
    batman-adv

    PRINC-347-01
    fw-ng-r376
    batman-adv

    PRINC-347-02
    fw-ng-r376
    batman-adv

    PRINC-347-05
    fw-ng-r376
    batman-adv

It's difficult to test definitively whether the 123 voucher no longer works because it's difficult to definitively "logout" of TeacherNet (although, in theory, http://logout should work), but it appears that the voucher has been disabled by one of the above actions.

Mitigating the Issue

To doubly-insure that the 123 access code would no longer work, I turned on the "WPA Password" option in the Cloudtrax dashboard to enforce the need to an additional shared WPA password before access to the network was permitted; I provided this password to the principal for circulation to staff.

I visited the school the same day and confirmed that this was working as expected.

I further confirmed that all Open Mesh Access points had automatically upgraded themselves to new firmware, fw-ng-r453.

Getting to the Bottom

To get to the bottom of the issue, I filed a support ticket with Open Mesh, the maker of the wireless access points and the maintainer of the Cloudtrax system:

We run a voucher-based Open Mesh system in an elementary school, network name "prince street school".

I received a report today that students were accessing the network using voucher code '123' and I was able to confirm that this did, indeed, work.

However, there is no such voucher for our network.

If I to CREATE a voucher 123 in Lobby Assistant I cannot -- a random voucher is created instead.

Please advise as to how I can remove the possibility of a login via '123'.

And the followed up with:

Some follow-up: attempting to use the voucher code '123' from a brand new device results in error message that the voucher is not enabled for this network if I uncheck "Vouchers work on all networks".

If I check " Vouchers work on all networks" then the voucher error is that the maximum number of devices for the voucher has been reached.

I have checked all of the other networks associated with our master login and cannot find a voucher code 123 in any.

And finally, after some additional research by Ken Williams:

Another follow-up: even though voucher code 123 does not appear in our voucher list in Lobby Assistant, I have found the I *can* see such a voucher with:

https://lobby.cloudtrax.com/vouchers/vouchers2.php?v=123

Why would this voucher not appear in the list? How can we delete it?

Open Mesh support was helpfully quick to reply:

I've passed this onto our dev team as a bug. This shouldn't be a valid voucher to use if it doesn't exist in your network.

I would reboot your AP's to erase any saved authentication information so your currently authenticated students are no longer authenticated. I've removed it.

The effect of unchecking the "Disable Automatic Upgrades" box earlier in my investigation had the effect of making each of the nodes in our network reboot, and so any "saved authentication information" was erased as part of that process.

I sent a further follow-up to Open Mesh:

Do we have any way of telling how the voucher got there in the first place?

And do we have any way of knowing why changing the "vouchers work on all networks" setting affected this voucher?

Finally, will a firmware upgrade (which we completed last night) have the effect of removing authentication data cached in the nodes?

To which they replied:

I believe it may be a voucher created a while ago by another user that was not in your master account. This is a glitch in the system.

Yes the firmware upgrade should reboot the nodes and erase that auth information.

At this point I closed the ticket.

Lessons Learned

  1. Using a managed third-party authentication platform has risks associated with it.
  2. My longstanding assumption that Cloudtrax and Open Mesh had "no support" was wrong: their support was quick and helpful, and addressed the issues.
  3. We were able to respond quickly to the issue using only volunteer support.
  4. We learned that authentication information for Cloudtrax vouchers is cached inside individual access points, meaning that if the vouchers are later deleted those holding the vouchers will still have access under the access points are rebooted. This is important to know when we consider how to manage vouchers.