The simple, one-word answer is: Encryption. Rich Shockey pointed out in our Virtual Event last week that SIPconnect, the SIP Forum's attempt to get consensus on SIP trunking implementation, mandates Transport Layer Security (TLS), which uses encryption. The problem, Rich notes, is that, "TLS is not broadly implemented yet," and would require an update of the SIP stack by developers, and hardware upgrades by some vendors to support the encryption.
Furthermore, there's the question of what to encrypt: Just the signaling, or the media payload (i.e., the voice packets) as well; encrypting the media would require Secure Real Time Protocol (RTP). The discussion that grew from Matt's No Jitter post centered around concern over people eavesdropping on conversations that run over SIP trunks; that suggests it'll be necessary to encrypt not just the signaling, but the media, too. One of Matt's commenters, Rob Wellbourn, is skeptical about this happening: "Good luck getting service providers to supported encrypted voice streams over their SIP trunks; the crypto overhead on their SBCs [session border controllers] is significant. (And remember that Secure RTP is encrypted hop-by-hop, so the service provider is always going to decrypt it at their SBC.)"
Up to now, enterprise VOIP security was pretty much an internal matter: You worried about somebody attacking your enterprise network-maybe hacking into your IP-PBX, maybe getting onto your internal IP network, maybe bringing down the whole shebang, voice and data, via a denial of service attack. Fortunately, the more exotic and dangerous-sounding attacks didn't materialize to a great extent, and denial of service remained the security issue that most VOIP managers had to worry about.
But if you go with SIP trunking, you're bringing another player into the equation, i.e., the carrier, and that always creates new questions and issues. And bringing in the issue of encryption, with its potential overhead, means the security problem bleeds into the quality of service problem, which in turn bleeds into the bandwidth problem.
How will security concerns ultimately affect SIP trunking adoption? My own guess is that if the myriad interoperability issues that beset SIP apart from the security realm can be addressed--and this is a big "if"--security concerns won't stop implementation. Until there's a highly-publicized incident. That's how security tends to play out.
In the meantime, those SIP interoperability issues aren't trivial, as Alan Percy has spelled out in a series of No Jitter posts (here and here). I encourage you to check out these posts as well as the very detailed, well-thought-out post by Matt and responses in the Comments.The basic issue seems to be that a SIP trunk is an IP connection, and any IP connection can be sniffed pretty easily.