Off the Kuff Rotating Header Image

Department of Information Resources

TxDOT hit with ransomware

Not great.

Texas’ transportation agency has become the second part of the state government to be hit by a ransomware attack in recent days.

On Thursday, someone hacked into the Texas Department of Transportation’s network in a “ransomware event,” according to a statement the department posted on social media Friday.

The departments’ website says some features are unavailable due to technical difficulties, but it is not clear what functions were affected by the attack. Agency officials did not respond to emailed questions Sunday.


Upon detecting the hack, staff at the transportation department “immediately” isolated the affected parts of the network and “shut down further unauthorized access,” according to the statement. James Bass, the department’s executive director, said his staff is “working to ensure critical operations continue during this interruption.″ The hacks follow a ransomware attack of unprecedented size that hit more than 20 local governments in Texas last summer.

See here for more on the attack on the court system’s website. In 2019, there was a coordinated attack on the systems of multiple small cities and counties.

I can’t find much in the way of news on this, so here’s TxDOT’s statement, via Twitter:

Maybe these two attacks are unconnected – there’s not enough information, such as what type of ransomware was involved and what the vector for it was, for me to take a guess – but the fact that there were two such attacks in a short period of time on two state systems sure seems suspicious to me. If I were at the state Department of Information Resources, I would be very busy, and more than a little concerned, right now. KXAN, CBS DFW, and Bleeping Computer have more.

Ransomware attack on state court system

Not great.

Websites for the Texas court system were still down Monday after a ransomware attack late last week left the network temporarily disabled, according to the Office of Court Administration.

Officials discovered the breach early Friday and quickly shut down sites and disabled servers to contain it, the office said in a statement. The hack did not impact e-filing and other services, many of which have been transferred to the cloud in recent years, according to the office.

“At this time, there is no indication that any sensitive information, including personal information, was compromised,” the office said. It added that websites for local trial courts are still available online.

The office said it detected the breach early and has refused to pay any ransom. While the courts have moved increasingly to remote hearings amid the coronavirus pandemic, the attack was unrelated, according to the office.

Officials have not said when the system will be back online, but they have set up a temporary website and are working with law enforcement and the Texas Department of Information Resources to investigate the attack.

As the story notes, this is not the first time that Texas governmental entities have been targeted by ransomware. The first thing that TDIR will need to figure out is whether this was actually targeted, or just a crime of opportunity, perhaps the result of someone opening a phishing email. If you follow this sort of news, you know that ransomware attacks are on the increase around the world; here’s a prominent recent example. I’m sure the system will recover from this, and good for the OCA if they detected it quickly. We just need to up our vigilance and defensive measures to stay on top of this.

Here are the vetoes

Sunday was the deadline for Rick Perry to sign, veto, or leave unsigned all of the remaining bills from the regular legislative session. He had 1170 pieces of legislation awaiting a decision while he was busy gallivanting around the country. Yesterday, he finished the task, issuing a total of 24 vetoes, one of which was for a fairly high-profile bill.

Notable among the vetoed bills is HB 242, a measure that would have banned texting while driving.

“I support measures that make our roads safer for everyone, but House Bill 242 is a government effort to micromanage the behavior of adults. Current law already prohibits drivers under the age of 18 from texting or using a cell phone while driving. I believe there is a distinction between the overreach of House Bill 242 and the government’s legitimate role in establishing laws for teenage drivers who are more easily distracted and laws providing further protection to children in school zones,” Perry said in his veto statement.

State Sen. Judith Zaffirini, D-Laredo, who wrote the texting while driving ban, said she was dismayed and disappointed that Perry vetoed the measure. Legislators’ decisions can save lives, and she said the texting ban would have done just that.

“From my perspective there will be blood on his hands,” Zaffirini said. “Every time that we hear about a tragedy related to distracted driving … I hope that is forwarded to the Governor.”

Perry’s vetoes will also mean a couple more agenda items for lawmakers to accomplish during the special session. He nixed sunset bills that are necessary to keep the Departments of Information Resources and Housing and Community Affairs going.

HB 2608, the sunset bill for TDHCA would continue operations of the agency until 2023, but Perry argued “prescriptive language was added to House Bill 2608 that would impose a new layer of bureaucracy that makes unrealistic demands of the state, delay assistance to communities hit by disasters and duplicate disaster planning conducted by the Texas Division of Emergency Management.”

Perry also took issue with the bill’s reliance on federal disaster recovery funds and a requirement the state issue plans for how it would use those funds.

“I do not take lightly the impact this veto may have in potentially shutting down TDHCA over the next year. That is why I have asked the legislature during this special session to amend language in pending legislation to continue the operation of TDHCA,” Perry stated.

You can see Perry’s statements here and here. Of greater interest to me are the bills he didn’t veto, including the Texas Cottage Food Law bill SB81 and the TV recycling bill SB329. As for the TDHCA bill, I don’t recall that being added to the call for the special session, but there’s still two weeks left in the session so there’s plenty of time for it if it needs to be in. Any surprises in what did and didn’t get vetoed to you?

State officially begins the IBM termination process

Been a long time coming.

The state gave notice to IBM [last] week that it will terminate the troubled $863 million data center consolidation contract. The process could take a full two years as the Department of Information Resources finds companies to finish the mammoth job and they take over the merger of 28 state agency data centers.

In the meantime, the existing contract requires IBM to provide “termination assistance” by maintaining staffing and providing the necessary information to ease the transition, said Ed Swedberg, DIR’s deputy executive director.

This break-up was telegraphed months ago when DIR said it would restructure the project and seek new companies to do the work. The restructured project will be broken up into several smaller, more manageable pieces in contrast to the the huge IBM-led effort.

That rebidding process will hit a critical milestone in January when interested companies must submit their plans. The objective is to have the new companies selected in August.

Swedberg said that process is now far enough along that the state felt comfortable starting the two-year clock on dismantling its relationship with IBM.

The first signs of trouble were reported two years ago, and after an attempt to fix things in January, it all fell apart in a hail of fingerpointing, with the beginning of the end in August. Let’s have a moment of silence for yet another failed privatization project on Rick Perry’s watch.

Bye-bye, IBM

Better luck next time.

The Department of Information Resources appears to be giving up on IBM — once and for all. The agency isn’t formally terminating its contract with the information technology and business consulting giant, which was supposed to coordinate the data centers and disaster recovery operations of 27 state agencies. But state officials sent a letter to IBM [Wednesday] saying they have no other choice to rebid the contract because they believe the company has failed to meet almost all of its obligations.

May the next outsourcer have better success. I mock, because it’s easy and fun, but outsourcing is hard. There’s a million reasons, and a million ways, things can go wrong. Still, it’s important to keep the mockery in mind for the next time some state official makes a grandiose pronouncement about how much money an outsourcing arrangement will save. Betting against it is the more likely winner.

One more thing about outsourcing in general, from the Statesman story.

IBM ran into problems from the very beginning, slowing progress and fueling frustration among the agencies. IBM has laid responsibility for the persistent problems at the feet of the participating state agencies, in particular, the Department of Information Resources.

“Ceding control of their individual (information technology) environments in favor of a centralized, common system was (and continues to be) unpopular with the constituent agencies, and without strong leadership from DIR, those agencies not only failed to cooperate, but in many cases actively resisted the project,” IBM wrote in a letter last week.

The main conceit of this kind of project is that you can save money by centralizing and standardizing. And that’s certainly true, although in some ways it’s basically a tautology. If you force everyone onto the same desktop, and you force all of your server-based applications onto the same back end, you will certainly spend less money on your IT. If that means that some specialized applications that a handful of people used to do their jobs are no longer available, or if it means that some specialized processes that were used to manage or present data are no longer allowed, well, that’s just the cost of saving money. One size seldom fits all, but you can pretend it does if it makes the bottom line prettier.

Anyway. Here’s a more recent letter from IBM disputing what DIR has to say. Again, I don’t know who’s right or wrong in this fight. I strongly suspect both sides have some validity to their claims. What I do know is that any outsourcing project is hard enough if everyone works together well. When communications break down like this, they’re impossible.

The State of Texas versus IBM: IBM responds

In July, the state gave IBM thirty days to respond to various charges relating to its ability to fulfill outsourcing contract obligations. IBM has now given its response.

In a letter released [last] Friday, IBM executive Cynthia McLean defended the company’s performance and said it will continue to meet with the agency to resolve issues in the 7 ½ -year, $863 million contract with the Texas Department of Information Resources.


Instead of a formal response, McLean said IBM would meet with state officials and attempt to hammer out a plan.

“As you know, we do not agree that IBM is responsible for the problems that you outline in that letter,” McLean wrote, adding that the company nevertheless recognizes that the information resources department, “is dissatisfied with the current state of the project.”

IBM said earlier that the department delivered too few of the state IT experts it promised and has balked at pushing 27 leery state agencies into consolidating their computer, Internet, printing and mailing services into two privately managed data centers.

Once again, we have to wonder if this marriage can be saved. And perhaps if it’s worth the bother.

In 2005, the Legislature forced more state agencies into the privately serviced IT pool. The new effort was supposed to save the state $178 million from April 2007, when IBM took over from previous vendor Northrop Grumman, to August 2014.

However, last year Grant Thornton, a consultant hired by the department, estimated that the state saved only about $10 million during the contract’s first two years.

Our Republican leadership in this state fervently believes that the private sector can always do better than government. I guess that’s true if you use government run by them as the basis for comparison. It’s certainly a low enough bar to clear.

The State of Texas and IBM: Not getting any better

No group hugs are expected any time soon.

Agencies that help Texans renew their automobile registrations, draw unemployment benefits and apply for food stamps and Medicaid face crushing demands – and IBM, the technology contractor for those agencies, isn’t even providing mundane services, a top state technology official testified today.

“We’ve experienced significant service delivery problems” that force state employees to wait many days for routine help with computer matters, said Ed Swedberg, a deputy executive director at the Department of Information Resources.

Speaking of a troubled, $863 million state contract with IBM, Swedberg described to a House budget panel “a major backlog of work requests” that, he said, have gone unheeded by the contractor.

“These are day-to-day requests, such as adding memory to a server, restoring a file or re-setting a password,” he said. “This is of course frustrating … and more importantly affects the agencies’ ability to serve citizens and other constituents.”

An IBM spokesman responded that the state is to blame for any problems.

“We are looking at each of [the department’s] numerous shortcomings since the very beginning of the contract,” said IBM spokesman Jeff Tieszen, who dismissed Swedberg’s testimony as simply more “misguided accusations” from the state.

See here for the previous entry. Did I mention that this has “lawsuit” written all over it?

The State of Texas versus IBM

It’s never a pretty sight when an outsourcing relationship goes bad.

IBM Corp.’s $863 million data center consolidation contract with Texas is teetering on collapse.

Seven months of negotiations aimed at righting the troubled project and salvaging the partnership fell apart at the end of June.

On Friday, the state gave IBM 30 days to fix the myriad problems that have plagued the effort to merge the data centers of 28 state agencies into two upgraded and secure facilities.

Turning around the mammoth project in a month will be a formidable task for IBM because some of the problems have been known for years and still persisted. Many industry insiders expect IBM and the state to part ways.

Karen Robinson , executive director of the state’s Department of Information Resources, provided IBM a seven-page litany of alleged contractual violations and “chronic failures.”

For example, Robinson said IBM had abandoned its obligation to provide enough people to do the work outlined in the contract.

IBM had reduced the personnel in one key project area from 124 in October to 40 in June. That pullback, in part, has brought the merger process to a virtual standstill.

The original contract set December 2009 as the completion date for the transition. So far, less than 12 percent of that work has been completed.


Jeff Tieszen , a spokesman for IBM, said the company “has fulfilled its obligations under the contract and today’s action by DIR was unnecessary and unjustified.”

“IBM very much regrets the state’s action and will aggressively protect its interest going forward,” Tieszen added.

Tieszen would not comment beyond the terse statement.

I don’t know about you, but I smell a lawsuit coming. As noted these problems are not new, though as recently as six months ago it looked like IBM was going to get another chance. So much for that. Just remember, when all is said and done, the point of this exercise was to save the state money. Unlike some other privatization fiascoes I could mention *cough* *cough* Accenture HHSC TIERS *cough* *cough*, this project needn’t have been controversial or on the fast track to Failsville. There’s nothing particularly unusual about data center consolidations. I don’t know enough to be able to say how or why it all went wrong – this would make a great topic for one of those sweeping Texas Monthly investigative stories, Jake; I’ll bet the Trib could do a bang-up job, too, not that I’m hinting or anything – but from what I can see it looks like individual departments had no choice but to participate, and if the state is to be believed, IBM didn’t have enough employees working on it. There’s nothing unusual about the former, though that doesn’t make it a good idea, but the latter sure is curious. Anyway, as I said before, never underestimate the potential for a Rick Perry-initiated privatization process to get screwed up. The DMN has more.