The Internal Revenue Service has abandoned plans to force taxpayers to a private facial-recognition system to access their online accounts, the agency announced this week. But critics say the now-foiled effort is part of an ongoing expansion of the frightening federal agency into more of our lives.

In a statement announcing it is dropping the program, IRS Commissioner Charles Rettig said, “Everyone should feel comfortable with how their personal information is secured.” Based on the reaction, few Americans were comfortable giving the feds a face scan.

According to Emory University Professor Usha Rackliffe, taxpayers using IRS.gov to pay their taxes or check on their balance would have to upload a video of themselves, along with a government-issued photo ID. Under the original plan, that would have to be done by the summer of 2022 to access older tax returns and information regarding the federal Child Tax Credit. It would not, Rackliffe said, impact the process of filing a tax return.

Rackliffe has serious objections to the IRS’s use of the technology, as do civil liberties organizations like the American Civil Liberties Union and the digital advocacy group Fight for the Future.

“The IRS’s plan to use facial recognition on people who are just trying to access their tax information online was a profound threat to everyone’s security and civil liberties,” Fight for the Future’s Caitlin Seeley George said in a statement. “We’re glad to see that grassroots activism and backlash from lawmakers and experts has forced the agency to back down.”

While members of Congress ranging from liberal Sen. Ron Wyden (D-Ore.) to conservative Sen. Ben Sasse (R-Neb.) spoke out against the $86 million contract, the entire New Hampshire delegation has been silent on the subject. However, the small-government folks at Americans for Prosperity- New Hampshire released a statement.

“When it comes to the use of facial recognition technology, government needs guardrails. Potential abuse of this technology has the potential to create a panopticon of constant surveillance of everyday people,” said Deputy State Director Ross Connolly.

Interestingly, one civil rights figure who doesn’t object is Rackliffe’s fellow Georgian, former UN Ambassador Andrew Young.

“I am on the other side,” Young told InsideSources. “I am for anything that is going to make people more secure and more free. I think facial recognition software is already used in just about every government office.”

He went on to say he is for photo (or video) ID in nearly every situation.

“We could have a voter ID with a picture on it. All the president would have to do is make an executive order. You could even put fingerprints in it. It can work for other things too. Whatever you sign up for, take a picture. It would only be a problem for those with a criminal record, but that is already a problem.

“India does it. That is a country with over a billion people,” Young added.

For Rackliffe, there are three areas of concern.

“The first concern is the accuracy of the software. With technology always comes challenges. The facial recognition software uses biometric scanning. So, it uses artificial intelligence and machine learning to validate you are who you say you are. It sounds good in theory, but in practice, studies have shown that if you have a darker skin tone, perhaps it may not always be accurate in identifying you. With the false negatives and false positives, the rates have been higher with people of a darker skin tone.”

Rackliffe is also uncomfortable with the IRS relying on an outside vendor like ID.me.

“This is a third-party company that you have between yourself and the government,” Rackliffe said. “That always makes people nervous. This is a company the government has opted to contract with to collect and keep that information. This can make people nervous because we often hear about companies being hacked and sensitive information being compromised.”

And, she notes, neither the IRS nor ID.me have laid out the nuts and bolts of their collection process or how information is stored.

“There are all these questions and concerns about who will have this information and what are they going to do with it. You could look at this as one-on-one face identification, like an iPhone. But there is a larger concern that it might be a facial recognition database. There are many questions like, will the information be given to law enforcement? These questions come up because the IRS has not come forward with who will have access to this information or how it may be used.

“We can all agree the use of technology to catch a bad guy is a good thing,” Rackliffe said. “But when does it become government overreach?”