Last week I wrote about the firmware upgrade to the new Airport Extreme base station that disabled the device’s best feature: end-to-end IPv6 routing out of the box. Today, James Woodyatt dropped some pointers for IETF working group discussions about this. They paint a very interesting picture about the way this great feature became a vulnerability and how it was then “fixed”.
Here’s the timeline as I understand it:
31 January 2007. Apple ships the new Airport Extreme base station. It is the first Apple router shipping with IPv6 routing and tunneling enabled, assigning globally routable IPv6 addresses to all devices on the local network. This allows Internet connections between Macs without any NAT-induced complications.
14 February 2007. Ars Technica is the first to note that the new IPv6 default configuration allows incoming traffic to devices on the local network— something that older, IPv4-based NAT systems prevent as a side effect of their designs.
8 March 2007. U.S. Department of Homeland Security issues a high-severity “Cyber-Alert” about the issue, claiming that the Airport Extreme design is remotely exploitable and “provides user account access” and “allows partial confidentiality, integrity, and availability violation”. News stories about the issue start to appear.
12 March 2007. Apple needs to do something about the growing media attention the issue is attracting. James Woodyatt of Apple’s Airport Extreme team is pointed to an IETF draft about protecting local IPv6 networks that recommends an implementation different from Apple’s:
“To implement simple security for IPv6 in, for example a DSL or Cable Modem connected home network, the broadband gateway/router should be equipped with stateful firewall capabilities. These should provide a default configuration where incoming traffic is limited […]. There should also be an easy interface which allows users to create inbound ‘pinholes’ […].”
Even though the draft is for an “Informal” RFC, as opposed to a “Best Current Practices” RFC that is usually used for implementation recommendations, the draft is interpreted by Apple executives as an official recommendation. Woodyatt writes to the IETF working group mailing list, asking for changes to the document to clarify that these are not official recommendations from the working group. It seems that despite growing pressure, Woodyatt is still hoping to be able to retain the suddenly-controversial feature in Airport Extreme.
29 March 2007. After a discussion on the mailing list, it becomes obvious that the NAP draft does actually represent the working group consensus. Others in the group are not seeing any problems with the recommendation to block incoming traffic. Woodyatt apologizes to the list and realizes that the only available course of action is to limit incoming IPv6 connections by default in Airport Extreme.
9 April 2007. Apple implements the new default behavior in a a new Aiport Extreme firmware. In two separate messages, Woodyatt writes:
When the U.S. Department of Homeland Security issues a CyberAlert, and when a Google News search on “ipv6 airport” returns fifty technical publications announcing that you have shipped a product that exposes networked computers to remote attack over IPv6, and you have senior vice presidents demanding a prompt response to all these security advisories that have put the trade press into a frenzy, then I’d like to see how far you get explaining that “some half-informed people will yell” but they should just be ignored.
I’m still a little unclear about the rational arguments in favor of this behavior, but I’ve learned to stop asking dumb and annoying questions and to just do my job.
The perceived security risk is gone, but there is now a problem with the Airport Extreme as an IPv6 router: out of the box, it does not work very well. Many network protocols and applications, such as iChat, BitTorrent, FTP and IPSec require two-way communication that is not possible after the change. In IPv4 routers, the workaround is to use protocols like UPnP and NAT-PMP. But they don’t work with IPv6, and there is no alternative — the problem didn’t exist in IPv6 before the decision to block incoming connections.
Woodyatt writes to the list again, arguing that since there is consensus for blocking incoming connections, a way to punch “pinholes” into that block is needed. But others still don’t see the problem with just opening the pinholes manually. It is simple to do, as Cisco’s Fred Baker demonstrates:
ip access-list permit ftp any host ftp.example.com
ip access-list permit ftp-data any host ftp.example.com
And this is where the issue stands today. In all likelihood, Apple will end up extending the NAT-PMP protocol to cover IPv6 — something it was never intended to do. It would be great to have a real standard for this, but I’m not holding my breath.
Again, it seems Apple alone understands the need for networking standards designed to work out of the box, without expert administrators.
Is there a way to turn off this new "feature?"
I was really excited when Ars reported that it had IPv6 capabilities, and frankly still do not understand why the router shouldn't expose the computers on the network. Doesn't IPv4 work the same way, that a firewall of some sort "should" be placed between a router and computers but is not required? Why would they _mandate_ some sort of firewall or NAT for IPv6? Given the gargantuan # of addresses available, bots would take years just to find a single accessible address.
But then, I'm preaching to the choir here.
Yes, this is only about the default configuration. You can still enable incoming IPv6 connections from the Airport Utility (but it's unlikely that 99% of users will).
Well, I'll have to investigate further once I convince my extended family to purchase it. I'm excited for this end-to-end IPv6 functionality without having to traverse NAT.
Huh. I just did the firmware update yesterday, and incoming-IPv6 functionality is still enabled. I wonder how my base station avoided this change.
Unholy mother of uninformed panic, this is so bafflingly idiotic. End-to-end NAT-less communication is the very *dream* that I hoped that IPv6 would bring, but now this single feature is deemed a *security issue* and recommended to be blocked? ACK!
The genius at Homeland Security who started this oughta be banned from ever using a computer again. He/she obviously takes way too much pleasure in making using a computer a pain.
I don't see the big problem. It's just a firewall, right? You will still get a globally routable address, right? Having firewall enabled by default would, IMHO, only lead to that the "noobs", who probably dont eaven know what a firewall is anyway, would get some extra security while the more advanced users can open up their router.
Might be a bad thing if apps such as iChat, that would possibly be used by the "noobs", _requires_ that remote hosts initialize a connection.
(by "noobs" i refer to the older generation, like my mom and dad for instance)
While NATs are a PITA, they do have the convenience of protecting potentially insecure clients from attacks that are external to the network.
It is critical that network vendors accept that _ALL_ network touching software and hardware has the potential for security flaws which expose those clients to attack.
The only mitigation is defense in depth, and for casual users preventing general access is the number 1 action that can be taken.
What mitigation does IP-6 provide, such that I would trust all of my home clients to be network facing?
So Patrik, every time you download a new application that wants to do something CRAZY (like oh my god, an audiochat, or a videochat, or file transfer--heaven forbid!) that requires pinholes you find it perfectly acceptable that you have to go and reconfigure your router to allow the port through?
How many people go out and update their routers' firmware on a regular basis? How many people that this is SUPPOSED TO HELP go out and update their routers' firmware on a regular basis?
There are so many things that are made simpler by keeping firewalls on the end nodes instead of routers. That's part of why IPv6 was (originally) designed this way.
@Andy Peters: existing configurations will retain their current
settings when updating the firmware. The new firmware only changes
the default, out-of-the-box, hard-reset configuration.
This is actually turning what was originally a hack (NAT and port forwarding in situations when only one *real* IP address existed) into a security feature, as if it was always the point.
If, from the beginning, users hadn't needed NAT and port forwarding because it worked transparently then this wouldn't be an issue today.
So that there is the original "Turn a flaw into a feature" that now bites us in the ass. Now they're making it look as if broadband routers come with NAT and Port Forwarding for security reasons (they don't).
Incidentally, back in the modem days the responsibility for security was on the computer's shoulders, not on the modem.
Of course, changing this ideology would mean convincing network experts artificially maintain their hold over a lot of places they don't really need to, nowadays (and I say this as a network expert myself, one who could do with less configuring of friends' routers).
Well... I don't know...couldn't you argue similarily that you don't need Firewalls on by default, because:
- only services running should be services you want/need
- a modern application will open its own port in the firewall anyway
- if applications can open the ports, then trojans can to.. so what is the point?
So what we really need is that port-facing services are sandboxed in such a way, that any bugs in these services can *never* spill over into the operating system.
Or where am I getting it wrong here?
julian: No, I see myself as an advanced user so I would most likely be able to deactivate my firewall.
I think that with a good UI it would be easy for anyone to open up ports for their apps. Imagine just browsing to http://myfirewall/ and the first screen that comes up would be:
"Hi! Someone has tried to connect to you. Do you want to let someone in?" followed by some addresses of recent connections and/or hostnames and allowbuttons.
It's an Apple product! I'm shure they'll come up with something nice :)
Does this firewall recomendation include outgoing connections aswell?
OK, I'm missing something here. As I understood it, IPv6 is not firewalled by the base station. But it does pass (in original firmware) all (?) incoming connections to the internal machines. How is this [i]not[/i] a security issue?
Yes, NAT is not a security feature, and shouldn't be used as such. However, there should be some kind of packet inspection going on--perhaps by a separate device.
I've had my web/email server on the internet in some form or the other for the last 12 years kuck nakes on the internet.
Same goes for all of my home computers.
I have never once had any of my home computers or servers hacked successfully, attacked successfully, or otherwise infected with a virus.
I have no concept of why those who have had these issues affect them continue to use the software that allowed it to happen. It wasn't the network's fault, it was your horrible OS or your horrible software that was at fault.
Passing IP packets to a computer is a FEATURE of the network, not a problem. Passing IP packets to a computer is not EVER a network security issue.
If your computer's operating system is so poorly designed to the point where getting network traffic is a security problem, my suggestion would be to stop using that operating system and use one that does not suck.
Please check out OpenBSD, FreeBSD, Mac OS X, Mac OS 9, and other operating systems which seem to have no problem riding the waves of the internet with incoming packets, and no problems.
Ken Carlile said:
IPv6 is not firewalled by the base station. But it does pass (in original firmware) all (?) incoming connections to the internal machines. How is this [i]not[/i] a security issue?
a. IPv6 is a protocol, not a piece of hardware that can be affected by a firmware update.
b. How can passing incoming connections to a computer is not a security issue is because that's how IP is designed... to pass packets from one IP to another. If my OS is properly designed, if my computer gets a malformed data packet, the OS should disregard the packet. My application should reject the packet if its got improper information. Its called the OSI model, and each layer of it is supposed to do a job... and for the love of God, its NOT the job of the Data Link Layer to inspect for anything other than good or bad packets. If the packet is legal, it should be passed. If the data is bad, the OS or application should reject the data in the packet.
The problem is that too many MSCE's and not enough computer scientists are involved with all of this stuff. Its not the job of your router to prevent your stupid-ass Windows machine from getting owned by an attacker - that's Microsoft's job. And just because they do a shitty job at thier job, that doesn't mean that users of other, well designed operating systems should have to pay a penalty. OpenBSD, by default, has a SINGLE port open - 22. You can beat the living shit out of it for weeks, and you will get absolutely nowhere - because it was properly designed by default.
Just because Microsoft has every port in the world turned on by default (for years and years... which they are slowly correcting), doesn't mean that the whole world must suffer.
The real solution to all of these network attacks on the internet is simple - remove the Windows machines, and things will work as they should work. Almost every other OS seems to handle the application-layer problems just fine, with occasional bugs, which are fixed rapidly.
Apple should NOT have to cripple their products and the internet ingeneral just because much of the world is used to poorly designed products from Microsoft. Any Linux, BSD, or Mac behind the original settings would suffer little to no real-world problems.
the_other_steve_jobs said:
Its not the job of your router to prevent your stupid-ass Windows machine from getting owned by an attacker - that’s Microsoft’s job.
It is the responsibility of everyone who produces network capable hardware to acknowledge, at a fundamental level, that both their, and others, products will have bugs and design flaws. It is inevitable.
The question now is, what is the appropriate way to design networks in light of this consideration? You can just blame someone for using the wrong OS. It is not the goal of end users to understand how networks operate.
Regardless of whatever supposed security concerns people have, one of the overarching goals of IPv6 was to bring back end-to-end to the Internet.
NATs are the exact opposite of having real end-to-end, and this decision goes against the very fundamentals of the design of IPv6. This decision has repercussions on aspects of protocol design that haven't been realized by many parties yet.
If you look through the linked threads you'll start to get a feel for the true scope of this decision...
Note that there's nothing like NAT-PMP or UPnP for IPv6. Both of those protocols were designed with the thinking that NATs are fundamentally broken and IPv6 would fix it all. Now it won't.
People need to stop thinking of the short term--just because you're used to network security design in a world of NATs today doesn't mean that they're the proper answer for the Internet as a whole. The amount of problems NATs cause far outweigh their usefulness as a half-attempt at security. This is why IPv6 was designed to be end-to-end.
Further, this is all stuff that has been in IPv6's design for a decade now. Now is not the time to reverse such fundamental decisions.
The silly thing here is that now Apple wants to add a firewall with a protocol that would allow hosts to create holes when applications need them (like uPNP or NAT-PMP can do for NAT). But then if all services running on your computer tell the router to not block packets incomming to them, how is this different from no firewall at all? If an application can create a hole in the firewall when it needs one, so can a tojan, a virus, or a hacker once one of your applications gets compromised. This kind of "firewall" is no more secure than NAT using uPNP or NAT-PMP.
Do we really need to rebuild the complexity of NAT only to have some illusion of security? I think it's pretty useless.
Exactly. IETF needs to come out and say that the original operation of the AirPort Extreme (n) was intended and works as IPv6 was designed.
People need to get over this thinking that NATs are somehow saving them. As UPnP and NAT-PMP become more ubiquitous NATs become more worthless as a means of security. That's true in IPv4 and it's true in IPv6.
"IETF needs to come out and say that the original operation of the AirPort Extreme (n) was intended and works as IPv6 was designed."
I wouldn't hold out much hope that IETF will do that. Both "rough consensus" and "running code" exist at this point. Stateful packet filters are going to be a fact of IPv6 life for the next thousand years.
Does the OS X firewall support IPv6? I know there was some concern in the Linux world about enabling IPv6 by default when some of the host-based Linux firewalls didn't work.
God help me if I ever encounter a system set up by you, Mr. Jobs. Remind me not to consider ever giving you a credit card, social security, etc. You are aware that every OS has an exploitable hole, right? Hell, OS 9 passwords were passed in cleartext! When I was in school, I was able to browse the desktops of other people on the network because they had no passwords, or because their passwords were weak. (after I tired of this, I left a text doc on their desktops telling them how to secure the machine...)
Firewalls are not a Windows specific invention. Security problems are not isolated to Windows. Get a reality.
And I'm well aware that IPv6 is a protocol, not a hardware device.
I'm with Ken there. I've got Windows, Linux, Solaris and MacOS X experience, and it's a hell of a lot easier to close the ports on my router, for which the interface is transparent and I have some reasonable trust that it's showing me ALL the open ports, rather than have to track down how to do it on each OS, and figure out whether the OS is showing me all my options, or whether it's keeping some under its hat because it needs the ports open for something, like Windows does. Having the firewall built into the router is a Good Thing. Perhaps dumbing down IPv6 was the wrong thing to do, but so was releasing a consumer-targetted router which doesn't have a built-in firewall.
Running 'sudo ip6fw list' gives me a list very similar to 'sudo ipfw list'. I'm not too familiar with ipfw, so I can't really evaluate them. I find it odd that no udp services are listed, though.
I think that Apple did an interesting thing with their IPv6 support in the airport, and it is an admirable idea. But, Apple being Apple, there isn't a lot of clear security information available. I recall reading that they are using 6to4 to establish the IPv6 tunnel. How are the security issues of RFC3964 addressed, for example?
For a device like the airport base station, I think that failsafe - denying all inbound TCP and UDP - has to be the way to go today. Relying on a properly configured client device is just a bad idea, especially when IPv6 still seems like it is going through the teething process - see Windows' IPv6 vulnerabilities (MS06-064) or the recent openbsd remote vulnerability.
The process of opening firewall pinholes is already performed on a Mac through System Preferences. Providing the same interface through the airport admin/setup utility or extending nat-pmp doesn't seem like a big deal.
@Sam:
How can it be a hell of a lot easier to close ports on OS X when it ships by default with no ports open? Linux too, except perhaps for 22. It's the fault of Microsoft that they leave ports open in Windows, and the rest of us who want the holy grail of unified end-to-end addressing shouldn't be punished.
All right, it's arguable that a router should have a firewall, but NAT's firewall ability is an accident, a side-effect. One of the whole points of IPv6 was to make the NAT bug go away. Now they're reintroducing it and people are calling it a firewall like it's something good? Blech. Put in a real firewall, but don't mislabel a bug as one.
For those folks who are panicking about the potential for evil IPv6 malware to hit attack surfaces on their networks unprotected by a sledgehammer of a stateful packet filter in their residential gateway, I recommend reading draft-ietf-v6ops-scanning-implications before you go much further. Just because SPI was how you had to do this with IPv4 does not mean that's the best tool for the job in IPv6.
Thank you for bringing this issue to light. I had no idea about this. In fact, I purchased an AEBS the day after reading this article. I did it because I want end-to-end internet. This security "issue" is absolutely ridiculous.
Thus ends the dreams of I'm sure quite a few software engineers at Apple. These were the dreams that they could program something that required 2-way communication and that their AEBS customers would simply enjoy the functionality w/o futzing around with arcane ipfilter settings.
For those of you complaining that IP packets are being ROUTED (router, duh)...here's the answer.
http://brent.stephenscorp.com/images/firewall.png
Thread over.
This should end all this hoopla. http://myietf.unfix.org/documents/rfc4864.txt