Surviving a 0-Day: Our Battle with a FreePBX Exploit 

Sangoma has been on the downward spiral for at least 10 years, not sure why you only say the last 5 years.

Sangoma? Do the right thing? Nope. Never.

If that's unsuccessful may we fork it?

You have to understand that the faxing through Fax Station is far different than traditional copper lines. It's never been secure enough to compete with traditional faxing, and honestly it made me scratch my head as to why anyone would want it. All these issues that people have had with Fax Station is never going to go away. The technology is just bad in general, and to be honest it's always better to have a traditional dedicated (copper) fax line just for that one job. Ever hear of a traditional fax being hacked? Yeah me neither.

Yea - it's pretty interesting, and it relates to some custom stuff we had built. The FreePBX firewall (which uses iptables at its core), when disabled does not stop the iptables service, so you can't just monitor iptables status - that's not accurate. But, there is a huge difference in the number of iptables chains/rules when the FreePBX firewall is on or off - so we do a count of those iptables rules, and if it drops below a certain threshold, that creates an error condition in Zabbix/Grafana. There's a bit more to it than that, but that's essentially what we're doing.

How did you think to add that monitoring? That's definitely useful now that we know about the exploit, but I never would have thought of it beforehand. I would love to see a "best practices" video where you talk about the system-level signals that you monitor.

@@CoolerQ I'd love to see a video on how this is done so we all can benefit.

Chris Thank you for being transparent in this. It opens an eye on how Sangoma is treating Freepbx. I would love if you would do a video or two around the two monitoring software you use to monitor Freepbx and other items. Cheers Richard

Noticed that too, very cool ☺️

And possibly the only useful purpose for said device.

keep in mind, this is 2023 too!

What do you recommend?

It is like offering a reward to someone who finds your puppy, then when they bring the puppy to you, you don't want to pay....you better not lose your puppy again!

We always err on the side of the customer. We didn’t bill them for this.

Haha - thanks! Probably got it off of Redbubble given how it has started to deteriorate.

I don't know where he got it, but somebody's got to go back and get a s%*tload of dimes.

Um. Most of their devs left Sangoma. Including Andrew Nagy.

Agreed - which is why so few of our customers were actually affected. We prefer ZERO open access to the outside world (perhaps just locked down for the SIP trunk) - there are many ways to do this including FreePBX's built-in VPN. But there are just some cases where customers have to have ports open - such as when they have a large work from home user base who are on dynamic IP addresses. In those cases, the Responsive Firewall is really the only line of defense.

With internet exposure, you should lock down the pubic facing services in Linux with either systemd security primitives or containers, this will protect the host and limit the damage even if the service is exploited..

@@CrosstalkSolutionsI can’t see a reason why they would not just use a service like DuckDns so the IP could freely change but still be updated. I don’t miss working with Freepbx. Also limiting the attack ability with services like Unifi’s by geo-blocking all countries excluding the US limiting an attackers entry point to data center servers and private residential and mobile IP addresses. That is what I do at least and just VPN outside if I need to access something from another country. The amount of alerts I get from port scanning alone from Russia, China, India, Africa etc could easily crash a notepad log file. I love the video btw and have been watching for years. Possibly one of the largest reasons I stuck my toes in the MSP world. I would love to see a collaboration with you and @rossmanngroup on getting Freepbx actually open sourced to the fullest potential. I have been thinking about picking up some of Unifi’s phones just to mess around with a buddy, have you had any good experience with them?

It seems to me based on the hacker presentation like they would have to have access to the actual php page in order to do this. If a spoofed phone or actor could get onto that network and call the code where the php webserver is hosted then they could execute the exploit. I guess if a user was compromised who had access to the webserver or could spoof an IP on the network then this would also be an issue. (I'm doing a lot of guess work here so take all this with a grain of salt).

Well....they were. If you trust Sangoma and their fix for it! The vector Chris stated was through a phone app on FreePBX that hosted services on the network to phones. My guess is it was usually used to Provision Sangoma phones. The password for those phones (unprovisioned) was simply "Login" so all you had to do was find a MAC address (NIC Serial Number) that was actually registered with FreePBX and you were in with a few lines of code in php added/modified. All security on the FreePBX server could be bypassed because of this. The best thing to do is restrict only the ports you need on the internet (SIP port, and TLS) and block everything else. And on those ports exposed to the internet you should have IPS, SIP blacklists, and GEOIP filtering guarding them. In my opinion you should never just "trust" another companies product anyway. To directly answer your question. Yes, It COULD be harder. It all depends on how you have your LAN setup as well. If a local device on your LAN could be compromised then no this would be easy. This requires a fair amount of networking knowledge to secure. In general though you are probably fine unless you have things on your LAN hosting things out on the internet. Just be sure you are NOT publicly hosting the provisioning part of FreePBX on the internet and work on Securing those ports as mentioned above and you SHOULD be fine.

Yes ...this was a vulnerability caused by sangoma

Depending on how it works, buy one, and get a Mac address that u can use

sounded like a $$$ issue with maintenance and they want you on the paid version... though if they get a bad reputation I could see it hurting their paid part.

Well yeah, except one other problem......as stated in the video by the guy that found the exploit. Some of the code is NOT opensource. This means it's not a complete opensource project. I'm not even sure it would work without that closed code. BUT!....if someone picked it up and just hired developers to look at it and take care of what's needed they might be able to just....make an alternative to Sangoma's version of it.

@@user-dq8oq4or7e okey as you say: You will have to figure that out and see if it is possible to work around that. I though it was as opensource as PFSense for example and that has been forked.

Sounded like there is some closed source content, so perhaps not the simplest task… The bigger issue with forking this, is you have to spread the word to those that are using FreePBX, so they migrate away. Much easier to take over the project & provide new releases, as then existing & future users benefit from a properly maintained product.

Isn’t this exactly what IncrediblePBX did? Seemingly based on FreePBX but a lot of the commercial modules are stripped out but with other security minded features added in.

Yea - the whole DEFCON video is worth a watch, but I didn't want to be too deep in the details for this one.

You could have a factory install certificate with the MAC in the SAN

Probably at least a part of it.

The original creator still owns his code for FreePBX. Even the code he wrote for Sangoma. He's just really not wanting to walk on Sangoma as the came to an agreement a few years ago over the signing key for FreePBX. It's just easier not to kick a hornets nest. Hopefully Sangoma would consider giving it back to him or selling it back to him.

Asterisk is the engine for all of their commercial products so it is the one thing that does get focus. Note that focus is centric to what has a proper business case for any of their commercial solutions.

I don't believe that Sangoma has commented publicly on the bug bounty program being pulled. I would also love to hear an explanation about that.

It would be better to use an SBC (Session Boarder Controller). SIP proxies seem to cause more problems than they solve. You might also look into DPI/IPS (Deep Packet Inspection/Intrusion Prevention System) systems on firewalls. They look at network traffic and block or notify bad stuff based on rules you setup.

Easier said than done - in order to fork, you'd have to remove any reference to Sangoma, FreePBX, etc. and also unwind any development related to their commercial modules. Plus, you'd lose the branding.

@@CrosstalkSolutions This is true Chris, but have you possibly considered asking the community to help you do this? Vates did this with Xen Server and created XCP-ng which had phenomenal financial support from the community. Would you be willing to take this on if the community was behind you on it?

What would you say is the best other opensource PBX system

you didn't watch the video, did you? it's all in there

Have you tried watching the video?

Please watch the video before making a comment that clearly shows you didn't watch the video.

Firewalls cannot help you when the service you are allowing inside itself is compromised..

What if a company doesn't patch? Don't publish it?

@@schwingedeshaehers You could issue a warning, but certainly not publish the technical details on how to actually carry out the exploit.

@@davepusey and if they still not update? There are researchers, that had problems, because they reported a vuln, and they didn't publish it. Iirc more than a year later, they were raided by police, because the vulnerability was used, even if they didn't publish anything, and didn't use it.