It’s okay to shed a tear for frame relay

By David Rohde Posted May 17, 2012

Frame relay became so popular in the 1990s that it’s easy to forget how radical it was. Variable-length packets, very little overhead, lack of extensive error correction, a bit to mark some of the packets as “discard-eligible” in case of congestion – this was new.

A response to the emergence of fiber networks, frame relay freed enterprises from the need for economically wasteful private line networks for bursty branch-to-data center traffic – so long as network managers could trust such a “virtual” private network for performance and security. It was a big leap for many. Robust service level agreements helped.

This leap of faith – but one that big enterprises took when they heard customer success stories and saw verifiable network performance – is useful to keep in mind as enterprises face another scary network migration from POTS to SIP. And the inevitability of generational changes is now pointed up by one of those touchstone moments in the telecom industry’s unique process for signaling rites of passage – changes in the web-based carrier Service Guides.

Our language-spotters in LB3 and TC2 have found an April 30 Service Guide notice in which AT&T schedules an end to its frame relay and ATM services. The notice states that starting next month, AT&T will no longer offer frame and ATM to new customers. Thereafter AT&T will support frame and ATM only until the end of existing customers’ contracts, requiring them to go forward only on a month-to-month basis after expiration. The services will then be turned off for good on April 30, 2016.

Not surprisingly, AT&T’s phase-out of frame relay is more conservative than earlier notices by Sprint (which basically ran away from the service) and Verizon (which backed off frame relay in a plan similar to AT&T’s, but much earlier). In a way, this order of retirement is backwards, and explains much about what has happened to the wireline side of the telecom industry.

Sprint’s original claim as the No. 3 carrier grew out of its consumer-driven “You can hear a pin drop” long distance ad campaign (which was a clever way of telling customers they would get AT&T quality at MCI prices). But Sprint really cemented its hold as a credible, first-tier enterprise provider with its frame relay service. Sprint’s offer was very different than AT&T’s, giving customers the right to mark all of their traffic as discard-eligible, pay only for the access and the port with no reserved subscription rights (or “Zero CIR” in the argot of the day), and trust the Sprint fiber backbone to send everything through anyway.

Many were the times that AT&T threw its vaunted FUD (“Fear, Uncertainty and Doubt”) against the Sprint bids, trying to convince users they had to pay for subscription rights in the AT&T frame network’s “closed-loop congestion management” scheme to compete against the other frame relay users. When AT&T’s frame relay network failed for 36 hours in the spring of 1998 even as Sprint’s network stood tall despite the widespread use of zero CIR, Sprint had basically won the argument – a cheaper service and a more reliable network.

To this day many of the Sprint enterprise WAN relationships that we see, often as a significant second carrier in enterprises with either dual-network requirements or multiple strategic business units, are legacies of the days when it broke through as a frame relay supplier. Yet it’s AT&T that stuck it out the longest. AT&T recovered from its mistake when it realized that the largest enterprises really wanted the higher-speed ATM service at the data centers and frame relay at the branches, and developed the most widely deployed “FRASI” product (frame relay to ATM service interworking).

Any of those FRASI networks that haven’t migrated to MPLS will now have to do so, but AT&T’s ability to recover from its outage by emphasizing the ATM side of the technology more, and building up reliability (especially after the legacy WorldCom network also went down in 1999), ultimately won in the marketplace. The effects are still being felt today as at least the wireline market moves toward duopoly in the first tier of carriers.

The technology is close to a goner now, but the echoes of what happened with frame and ATM over the last 20 years, and who stuck with it for the most critical periods, are still being felt today. Unless you still have a lot of frame relay (and thus need to think about more critical considerations), hoist one for frame and ATM at your next opportunity. It was a great service while it lasted, and holds many lessons for the future.

AT&T’s Property Tax Allotment increase restarts the surcharge merry-go-round

By David Rohde Posted May 16, 2012

Here come more changes in the percentage-based surcharge line-up. AT&T has just increased its “Property Tax Allotment” fee from 3.17% to 4.05% of applicable revenues, effective May 1. More changes in the percentage-based federal surcharges, either from the other carriers in answer to AT&T’s property tax move, or from AT&T itself on other fees, are likely to come.

How does AT&T know that the property taxes it pays on all of its POPs and telco central offices, split among all of its customers, now comes to a rather amazing 4.05% of the kinds of revenues on which you also pay universal service surcharges?

AT&T doesn’t know that, of course, and neither does Verizon or anyone else. That’s a point on which this blog has had a lot of laughs going back to our earliest days. After all, with these property tax surcharges having about doubled in 3½ years, the implication is that commercial property has doubled in value nationwide since right before the financial crisis hit, which obviously isn’t true.

Yet AT&T could secretly claim a victory in my even attempting to justify the level of this surcharge. Doing so implicitly buys into the notion that ANY property tax surcharge makes sense if only we could determine its proper level! But of course that’s questionable as well.

So what’s really changing here? Two things, I think. First, as the non-USF percentage surcharges rise to more and more material levels, the interesting little “seams” in who exactly pays what become more important. There’s a little-known fact that some large AT&T customers on a contracting platform that goes by the acronym VTNS, which emerged from the original “Tariff 12” custom contracts of the early 1990s, do not pay all of the non-USF surcharges, and that means something.

Now I’m hardly suggesting that you should run to AT&T and demand a change in contract and service provisioning platform for this reason – that’s not realistic. What is important is that this exception is just one example of how an increasing divergence in how surcharges are specifically calculated – across carriers, billing platforms, and type of applicable services, NOT just the percentages themselves – must be accounted for in competitive bidding. Unless you know all of these variables, the attempt to account for the surcharge dollars in whatever you have out for bid is likely to fail.

Second, now is finally the time when the FCC is set to address the whole “contribution methodology” for universal service, as opposed to just looking at a range of USF distribution and intercarrier compensation issues, as it did in a very extensive ruling and report last year. Whether the rise of non-USF surcharges to almost silly levels is the last gasp of an old regime in which surcharges that have no connection to customer revenues are charged on percentages of those revenues anyway, or will be replaced by an entirely different system, is the focus of a key effort of LB3’s regulatory practice and depends on corporate user input as well. The stakes have been raised yet again – watch this space.

Short contracts, reverse auctions, buying consortiums, and other things that don’t work in telecom

By David Rohde Posted May 10, 2012

One reason that we’ve held a telecom negotiations conference twice a year for 20 years is that there’s a continual flow of very smart people into the telecom field who have to unlearn the right way to buy every other business product or service.

Things that make perfect sense in buying both low-tech and high-tech items for companies make no sense when buying telecommunications services. Actually, strike that: They make sense, but they don’t work. You have to make a mind shift when going into telecom procurement. An ability to suspend disbelief comes in handy.

Example: To many buyers, the best way to procure something is through a reverse auction. The problem in telecom is that you don’t know what exactly is being auctioned (or worse, the carrier bidders know but you don’t).

Think about it. In a straight auction, everyone can see the painting they’re bidding on, right in front of them. But in telecom, is a T-1 access line a T-1 access line? It is, provided there’s only a single, straightforward monthly recurring charge, and no hidden central office connection charges. Alas, that’s unlikely to be the case unless you pose additional questions and present a complete bidder worksheet that forces bidders to reveal (or waive) all their hidden charges.

Otherwise the winner of the reverse auction could be the carrier with the lowest MRC but with heavy connection charges that are stuck in their Service Guide – charges that they later claim they couldn’t do anything about because you didn’t reverse-auction those, too.

The backwards logic of telecom pricing and terms forced by the existence of Service Guides drives many other counterintuitive best practices. It bears continual repeating: If you don’t address a matter in your contract, then whatever the carrier’s ostensibly public (but often dense and confusing) Service Guide says rules.

That’s why appeals to short and simple contracts are misleading. LB3′s Justin Castillo gave a funny example when he noted how certain carriers pretend to be user-friendly by offering short contracts, but then stick URLs in them that incorporate pernicious items. And even without a lazy handoff in a contract to a web page, silence on a matter will default to whatever the Service Guide says.

Leverage, timing, contract commitments and an enterprise’s overall reputation for competitive bidding also explain why the natural logic of procurement consortiums – pulling together bigger and bigger demand sets for supposedly better prices – rarely works in telecom. The mere size of a demand set is overwhelmed in importance by other factors, such as the amount of uncommitted spend and the ability to make apples-to-apples comparisons. The very difficulty of comparing rate elements – just think about how AT&T and Verizon differentially treat class of service on MPLS – is magnified when you have multiple enterprises at the table.

Put it this way: If you’re reading this, you probably have the ability and interest to drive your own telecom destiny via a true demand set build, a focused RFP, and a genuine consideration of multiple carriers, especially in a migration to a new technology. Don’t let generalized procurement truisms that don’t apply to telecom’s unique technological and regulatory ecosystem push you off this path.

The sociology of the cell phone store, and what it means to you

By David Rohde Posted April 26, 2012

You probably do so anyway in the course of events, but if you haven’t visited a wireless retail store in the last six months, make an excuse to check one out again.

The last couple of times I’ve been in a Verizon store in Northern Virginia, the sense of stratification between two distinct customer groups was palpable. Of course the stores were very busy, and most of the people were in there signing up for cool stuff. But both times there were also a number of customers who came in with anxiety written all over their faces.

Almost as soon as they walked in, they would start pleading with the store staff: “Please kind sir, I really only want this for phone calls and to be reached in an emergency, please don’t charge me a lot of money, please oh please.” It was as if they knew that the moment they entered, they would be caught up in the vortex of mobile broadband communications and walk out penniless.

It didn’t help that the flip phones and other basic devices were tucked away in a corner, away from the sea of Android and Apple devices (with the BlackBerries in the back – which is another matter we’ll be addressing soon). But I think the bigger factor was the sociological pull of seeing sellers and buyers rapidly coming to an agreement about the basic outline of the transaction – full mobile functionality with voice, email and Internet, and access to thousands of apps – and steeling oneself not to be part of the crowd.

I bring this up to put in perspective a number of recent news reports about a supposedly surprising slowdown of the growth of the wireless market. The question is whether this really is surprising from a mass market perspective, and whether it has any meaning for the enterprise marketplace, where demands to support the entire range of smartphones and tablets, and to enter the world of “Bring Your Own Device,” are running amok.

As a global consulting firm, we at TC2 have learned to put generalized statistics about the United States in perspective. My colleague Joe Schmidt – who holds the distinction of having served TC2 clients on three different continents – long ago noted that cell phone penetration in South Korea was over 100% (meaning that everyone had at least one device and some had more). But that’s a country with essentially universal wireless coverage, high average population density, and less demographic diversity than the U.S. The idea that in America, the entire population isn’t going to come to the same conclusion at the same time, seems to come as a surprise to analysts.

Also in play is the classic statistical problem faced in IT industry market research: Is the year-over-year doubling of a market off a small base more important than the growth by 20%-40% off a large base? It will be if that’s what you’re counting on, but that’s for people in the vendor executive suite, not for people on the front lines, for whom continued growth off the large base is the thing that’s keeping them busy all day long satisfying customer or end-user demands.

From this perspective, what’s important is not the fact that some people aren’t trading up (which analysts should have seen all along) but their anxiety about not trading up that’s the key. The source of that anxiety is the same one you face in your job – overwhelming cultural pressure by everyone who IS trading up to support mobile broadband, with an end-user sensitivity to the selection of the telephony/data device that was never seen in the world of office desktop phones and PCs.

If your senior management team asks why your company’s mobile demands seem to be running counter to a supposed “slowdown” in the market growth, help them to understand why you and your end-user base – or at least the part of it that’s the subject of wireless procurements – are in the part of the stratified market that’s seeing little or no slowdown at all.

The looming spectrum shortage: A threat to telecom budgets or a scare tactic?

By Dorothy Nederman Posted April 25, 2012

The following is a guest post by TC2 Analyst Dorothy Nederman, who is based in Washington, D.C.

Here in Washington, where public policy and private interests clash every day, the big telecom players are adept at framing their own positions as the ones that happen to best serve the public interest. Nowhere is that more evident than in the FCC’s well-publicized concern over an impending spectrum shortage, tagged at up to 275MHz by 2014.

The carriers do not disagree with the FCC’s concern. In fact, they are happy to follow suit and warn that without an increase in the supply of spectrum, your wireless data bill is bound to skyrocket. But is that really inevitable?

Verizon Wireless has positioned its pending purchase of spectrum from four cable companies – including one deal with Comcast, Time Warner Cable and Bright House Networks, and another with Cox Cable – as a benefit to its mobile broadband customers. But its competitors, particularly the ones behind Verizon like Sprint and T-Mobile, call it a move to hoard spectrum that only makes things worse.

With unlimited data plans falling by the wayside and some data allowances being throttled, these hoarding complaints have gained traction. In the required regulatory process to transfer the spectrum licenses, Verizon has even offered to sell some of its existing 700MHz spectrum licenses in exchange for approval.

But it’s possible that the real problem is the way that Verizon – and everyone else – actually uses spectrum. Wireless industry pioneers cited in a New York Times article last week claimed that warnings of spectrum shortages and price increases might be simply an industry-wide scare tactic. Some of them even question the FCC’s data sources in predicting a shortage at all.

They go back to some fundamentals to ask why slices of spectrum are even assigned exclusively to one player at a time any more. Current devices, no matter how capable, continue to be manufactured to utilize only certain frequencies. But there are newer technologies, such as the smart antenna, that conceptually are able to make better use of transmitting signals, so that there is less interference between frequency bands. That means that “everyone could share spectrum and not run out,” as the Times hopefully puts it.

The immediate problem is that none of the wireless carriers – now pressured on one side by their parent companies to deliver the bulk of the earnings, and on the other side by the need to pay popular device manufacturers huge subsidies – have stepped forward to implement the solutions laid out by the industry pioneers. The benefit of this article is that it questions the carriers’ strategy while discussing technological devices to better manage mobile bandwidth demand, which could push the FCC to investigate mandating their use.

Check out the article and see who you think is correct here. Wireless data is certainly one area where enterprise users have to keep an eye on both current demand and future needs. As we follow the Verizon spectrum purchase story, and the fierce scramble by others for the capacity and money to stay competitive on mobile broadband, this is an important perspective to keep in mind. It will be interesting to see if the FCC and the carriers give it credence as well.

Vodafone does want Cable & Wireless (but shareholders want them to want it more)

By David Rohde Posted April 23, 2012

It’s Monday and Vodafone has indeed put in its bid at the deadline for Cable & Wireless Worldwide, at just over £1 billion, or about $1.7 billion. The only problem is that if that price looks low for such a venerable telecom brand – well, that’s exactly what C&WW’s largest single shareholder thinks, and that’s a big hurdle.

As of the end of the business day in London, investment firm Orbis, which owns 19% of C&WW, is in effect objecting to the deal by refusing to provide the formal “irrevocable understanding” that is common in British M&A deals. Vodafone’s offer may be almost double what C&WW was worth on the stock market before it put itself for sale. But it still comes to only 38 pence per share, following a string of financial and market disappointments for C&WW recounted by Reuters in its report today on the latest chapter of the company’s sale attempt.

Underneath the ongoing drama is a tension you often see in important telecom deals that fly just below the radar of obvious blockbusters (such as the U.S. mega-mergers of the middle of last decade, or the failed AT&T/T-Mobile merger last year). You have to judge between the long-term prospect of the wireless giant Vodafone and the wireline Cable & Wireless Worldwide fused together to provide global, strategic enterprise competition, and the more near-term considerations that are clearly in Vodafone’s actual calculations (such as owning more of its own fixed-line backhaul for its wireless network traffic).

At $1.7 billion, the price is either a steal based on the deal’s long-term synergistic potential, or a waste given C&WW’s longstanding problems in making itself fully relevant to international competition. No doubt Orbis wants Vodafone to look much more on the upside, but that can be hard to do when a company that’s been bleeding is on the sale block.

For me it’s fascinating to see Cable & Wireless yet again trying to achieve an outcome where it fulfills its century-old potential and becomes top-of-mind for U.S.-based multinationals. In the late 1990s, Cable & Wireless did the biggest single yo-yo in the entire telecom boom of the day – inserting itself in the whole MCI/WorldCom merger saga, picking up a global Internet network as part of a required divestiture, and making an enormous splash at exactly one of the giant “Interop” trade shows of that era, before dramatically turning course and retreating.

The names are different now, and under this 2012 deal, the whole enterprise would likely be branded going forward as Vodafone. And U.S. users have to remember that the company for sale here, Cable & Wireless Worldwide, is technically a spinoff of the entire Cable & Wireless empire, albeit the part that they would care about most. But the underlying facilities around the world, and the implications for pan-European, Asian (especially Indian), and prospectively North American enterprise contracts, are big. More to come.

Cable & Wireless bidders: And then there was one

By Ben Fox Posted April 19, 2012

The following is a guest post by TC2 managing director Ben Fox.

As of today there’s one bidder left at the table for Cable & Wireless Worldwide – Vodafone.

India’s Tata Communications has formally withdrawn from the C&WW bid talks, stating that the two parties had not been able to agree on a price. This leaves just Vodafone considering whether or not to make a takeover bid – one that would seem to be a good fit with Vodafone’s current global enterprise strategy. If Vodafone does go forward, the bid would likely be valued in the region of £1 billion.

A notable undercurrent in the UK to the potential takeover by Vodafone has been whether or not Vodafone would be able to take advantage of C&WW’s UK tax situation to offset its own UK tax bill. As the BBC’s Robert Peston explains, C&WW has a £5.2 billion UK “capital loss” on its books. The theory is that in acquiring C&WW, Vodafone may be able to use this to reduce its own UK taxes.

No one seems quite certain whether or not Vodafone would be able to offset its UK taxes using this acquired capital loss. But it’s not hard to read between the lines of Tata’s “inability to agree a price” and infer that C&WW must therefore believe Vodafone would pay more, and that the tax situation may be helping Vodafone to put a higher value on C&WW than Tata does. Tata does not have a large UK tax bill to offset, so the C&WW capital loss is of no value to it.

At one point there were even noises coming from the Vodafone camp that the Vodafone board was nervous about taking over C&WW because of the potential bad press from reducing its tax bill. Although frankly it’s not clear how sincere such sentiments were – Vodafone (like many large UK companies) has enough bad press in the UK about how it manages its tax bill already (not to mention a long term tax dispute in India), and doesn’t seem to be too bothered by it.

Regardless, Vodafone has until close of business today to make a bid for C&WW or not.

Additional note from David Rohde: Late in the day London time, Dow Jones reported that the deadline for Vodafone to make a bid for Cable & Wireless Worldwide had been extended until Monday morning. The importance of this developing story is second nature to telecom managers with global responsibilities, but for those U.S.-based telecom professionals whose enterprises are only now expanding their international reach, and who may only know the name Vodafone vaguely as a minority owner of the Verizon Wireless unit, I encourage you to enter “Vodafone” on the blog search, or click on the “Global” tag in the topics cloud, and you’ll see a lot of information on Vodafone’s important presence and challenges in both wireless and wireline, brought to us by Ben and our other London-based TC2 consultant, Mark Sheard.  We’ll of course continue to follow this latest development.

In-region or out-of-region: What difference does it make where an access circuit is located?

By David Rohde Posted April 18, 2012

It’s intuitively obvious that if AT&T or Verizon is going to sell you a dedicated access line to a national voice or data service, the price of the access circuit depends on whether AT&T or Verizon is also the local exchange carrier at that location. Right?

Except it’s not at all obvious. These core enterprise voice and data services have been sold by the classic AT&T national long distance business and the legacy MCI business within Verizon since long before they were part of any of the RBOCs. From those days up until now, these services have been sold in a national competitive market that AT&T and Verizon can’t excuse away in their bid responses by claiming that they only “control” part of the country.

And yet – experience and common sense hold that for certain demanding requirements, each carrier is just naturally going to be able to do better where all the incumbent local “plant” is its own. To take two extremes: For a national demand set of T-1 access lines, the flat (or nearly flat-rated) national price should apply equally in all locations. But for a list of 100M Ethernet access circuits, the carrier is virtually certain to give you a better price in its own ILEC territory than in another carrier’s ILEC territory.

So where’s the dividing line? What kind of bids should you receive for, say, T-3 access, or 10M Ethernet? Should there be one national flat rate? Two rates – one for “in-region” and another for “out-of-region” or “out-of-footprint”? Or should the bid be entirely custom – individual rates for individual locations?

This question gets to the heart of benchmarking telecom and network services, and why it’s much more than just looking at two numbers side by side. All other things being equal, a simple price schedule is better than a complicated one – but only if it actually produces a lower Total Cost of Ownership (TCO) for your current and projected demand set.

If you are ramping up the number of T-3 access circuits you need – often to accommodate MPLS ports in multiples of 5Mbps or in burstable high-speed flavors – you may see dual pricing for in and out of region. But both prices could be good … or both could be bad. Or the delta between the two price points could be acceptable … or too wide.

Alternatively, if you now need a meaningful quantity of 10M Ethernet access circuits to go with Ethernet ports on MPLS, Internet or other services – and many enterprises are only beginning to realize how widely available Ethernet-based carrier services have become – the ability of a carrier to commit to an objectively good price in the “other guy’s” territory may be more important than merely the fact that the price may be different than elsewhere.

There’s a certain analogy here with the surcharge questions my colleagues and I often discuss. Only full modeling that combines pricing, quantities, locations, and the carrier’s terms as reflected in its offer will unlock the mystery of a good offer vs. a bad one. For now, be aware that as you move up the bandwidth ladder, region-by-region pricing exists, whether it should or not. What you do with it is what matters.