Historically, the network topology at my home has been horrible.
A little background on why it’s so foul. Ever since I got ADSL, all the literature I’ve received regarding the set-up parameters has told me the encapsulation my connection uses is PPPoA with VC-Mux. PfSense and semi-decent routers like the WRT54G only have PPPoE/LLC clients, so I always imagined my resulting configuration was the only possible one.
With that, behold the abomination of a network I have used for a long time:
My convoluted network
Yes, I was double NATing. At one point I had such a terrible router with no DMZ capabilities that I actually had to forward ports on the ISP router to PfSense and then in PfSense to whichever tortured client was gasping for visibility on the World Wide Web (the www is the internet is the www rite?).
The router I have currently is a Telefónica-branded ZyXEL P-660R-D1. It’s a holdover from an ADSL line we had in another house, and after my WAG54G broke it was all I had.
The other day, for some reason I have now forgotten, I stumbled upon the fact that many PPPoA setups actually accepted and worked if configured as PPPoE. Peeing my pants in excitement I first changed them, and then set about confirming this new morsel of knowledge. Preliminary results proved that yes, in fact, I could connect via PPPoE! Oh the joy.
I set about configuring it in bridge mode, so as to delegate the PPP session initiation responsibilities to PfSense. Easy enough -I thought- select bridge from this here dropdown menu, choose NAT-none radio option here, click click bleep-bloop-done! Same in PfSense, set the WAN interface to PPPoE, put my username and password in, apply and voilà. Not so much. I couldn’t get the PPP link up, and perusing the system log I found lots of this:
May 16 15:55:36 router mpd: [pppoe] outgoing packet is demand
May 16 15:55:36 router mpd: [pppoe] IPCP: Open event
May 16 15:55:36 router mpd: [pppoe] IPCP: state change Initial --> Starting
May 16 15:55:36 router mpd: [pppoe] IPCP: LayerStart
May 16 15:55:36 router mpd: [pppoe] bundle: OPEN event in state CLOSED
May 16 15:55:36 router mpd: [pppoe] opening link "pppoe"...
May 16 15:55:36 router mpd: [pppoe] link: OPEN event
May 16 15:55:36 router mpd: [pppoe] LCP: Open event
May 16 15:55:36 router mpd: [pppoe] LCP: state change Initial --> Starting
May 16 15:55:36 router mpd: [pppoe] LCP: LayerStart
May 16 15:55:36 router mpd: [pppoe] device: OPEN event in state DOWN
May 16 15:55:36 router mpd: [pppoe] device is now in state OPENING
May 16 15:55:45 router mpd: [pppoe] PPPoE connection timeout after 9 seconds
May 16 15:55:45 router mpd: [pppoe] device: DOWN event in state OPENING
May 16 15:55:45 router mpd: [pppoe] device is now in state DOWN
May 16 15:55:45 router mpd: [pppoe] link: DOWN event
May 16 15:55:45 router mpd: [pppoe] LCP: Down event
May 16 15:55:45 router mpd: [pppoe] device: OPEN event in state DOWN
May 16 15:55:45 router mpd: [pppoe] pausing 6 seconds before open
May 16 15:55:45 router mpd: [pppoe] device is now in state DOWN
May 16 15:55:51 router mpd: [pppoe] device: OPEN event in state DOWN
May 16 15:55:51 router mpd: [pppoe] device is now in state OPENING
I spent a few hours trying to get it to work, changing one part of the configuration at a time, rebooting both routers just in case, yadda yadda. In the end I couldn’t even connect with my original PPPoA/VC-Mux settings. I didn’t have a working phone with me and had to wait until the next day to call my ISP.
Their logs showed me having 16 line-drops over a period of few hours, and seemingly one of the line-drops triggered the system to disable my account because they thought I was hacking them. Or something. The support guy wasn’t very clear on that.
With my internet restored (and my brother’s fingers unclasped from my neck) I decided to do some extensive searching on the issue. After much reading and about 200 Firefox tabs open, I found out that seemingly, with my router, Bridge mode only really gets activated if you do it through the Wizard. I’d just gone straight into WAN settings and changed it there.
Parallel to the research, during my tests the day before, I had noticed something strange on the system monitoring page. After negotiating the connection speeds, the router was alternating between two connection profiles; my one -aptly named ‘Jazztel’- and some other one named ‘ISP2′. I couldn’t find mention of ISP2 anywhere on the router’s web management pages, so I telnetted in. There I quickly found the rogue profile under ’11.1 Remote Node Configuration’. It seems the profile was pre-loaded onto the router by Telefónica and served some mystical purpose. Deactivating the profile broke my internet again. I thought the solution might be just deleting it so after much hemming and hawing I went for it. With my heart thumping I rebooted. Success! The internet still worked.
With that peculiarity sorted out, I went through the Wizard to set the router into bridge mode and saved the settings. Then in PfSense I configured the WAN interface as PPPoE again, with my username and password, and crossed my fingers. After about a minute my internet was working again and there was much joy in my heart.
New and improved network diagram
Over the next few days I’ll get to see whether all this trouble was worth it or not. Will PfSense need rebooting every other day? More to come.
Update: here are the conclusions after a few months, regarding the change.
