Reprinted with permission from ProPublica.

In violation of a longstanding legal mandate, scores of federal law enforcement agencies are failing to submit statistics to the FBI’s national hate crimes database, ProPublica has learned.

The lack of participation by federal law enforcement represents a significant and largely unknown flaw in the database, which is supposed to be the nation’s most comprehensive source of information on hate crimes. The database is maintained by the FBI’s Criminal Justice Information Services Division, which uses it to tabulate the number of alleged hate crimes occurring around the nation each year.

The FBI has identified at least 120 federal agencies that aren’t uploading information to the database, according to Amy Blasher, a unit chief at the CJIS division, an arm of the bureau that is overseeing the modernization of its information systems.

The federal government operates a vast array of law enforcement agencies — ranging from Customs and Border Protection to the Drug Enforcement Administration to the Amtrak Police — employing more than 120,000 law enforcement officers with arrest powers. The FBI would not say which agencies have declined to participate in the program, but the bureau’s annual tally of hate crimes statistics does not include any offenses handled by federal law enforcement. Indeed, the problem is so widespread that the FBI itself isn’t submitting the hate crimes it investigates to its own database.

“We truly don’t understand what’s happening with crime in the U.S. without the federal component,” Blasher said in an interview.

At present, the bulk of the information in the database is supplied by state and local police departments. In 2015, the database tracked more than 5,580 alleged hate crime incidents, including 257 targeting Muslims, an upward surge of 67 percent from the previous year. (The bureau hasn’t released 2016 or 2017 statistics yet.)

But it’s long been clear that hundreds of local police departments don’t send data to the FBI, and so given the added lack of participation by federal law enforcement, the true numbers for 2015 are likely to be significantly higher.

A federal law, the 1988 Uniform Federal Crime Reporting Act, requires all U.S. government law enforcement agencies to send a wide variety of crime data to the FBI. Two years later, after the passage of another law, the bureau began collecting data about “crimes that manifest evidence of prejudice based on race, religion, disability, sexual orientation, or ethnicity.” That was later expanded to include gender and gender identity.

The federal agencies that are not submitting data are violating the law, Blasher told us. She said she’s in contact with about 20 agencies and is hopeful that some will start participating, but added that there is no firm timeline for that to happen.

“Honestly, we don’t know how long it will take,”Blasher said of the effort to get federal agencies on board.

The issue goes far beyond hate crimes — federal agencies are failing to report a whole range of crime statistics, Blasher conceded. But hate crimes, and the lack of reliable data concerning them, have been of intense interest amid the country’s highly polarized and volatile political environment.

ProPublica contacted several federal agencies seeking an explanation. A spokesperson for the Army’s Criminal Investigation Command, which handles close to 50,000 offenses annually, said the service is adhering to Defense Department rules regarding crime data and is using a digital crime tracking system linked to the FBI’s database. But the Army declined to say whether its statistics are actually being sent to the FBI, referring that question up the chain of command to the Department of Defense.

In 2014, an internal probe conducted by Defense Department investigators found that the “DoD is not reporting criminal incident data to the Federal Bureau of Investigation (FBI) for inclusion in the annual Uniform Crime Reports.”

ProPublica contacted the Defense Department for clarification, and shared with a department spokesman a copy of the 2014 reports acknowledging the failure to send data to the FBI.

“We have no additional information at this time,” said Christopher Sherwood, the spokesman.

Federal agencies are hardly the only ones to skip out on reporting hate crimes. An Associated Press investigation last year found at least 2,700 city police and county sheriff’s departments that repeatedly failed to report hate crimes to the FBI.

In the case of the FBI itself, Blasher said the issue is largely technological: Agents have long collected huge amounts of information about alleged hate crimes, but don’t have a digital system to easily input that information to the database, which is administered by staff at an FBI complex in Clarksburg, West Virginia.

Since Blasher began pushing to modernize the FBI’s data systems, the bureau has made some progress. It began compiling some limited hate crimes statistics for 2014 and 2015, though that information didn’t go into the national hate crimes database.

In Washington, lawmakers were surprised to learn about the failure by federal agencies to abide by the law.

“It’s fascinating and very disturbing,” said Rep. Don Beyer, D-Va., who said he wanted to speak about the matter with the FBI’s government affairs team. He wants to see federal agencies “reporting hate crimes as soon as possible.”

Beyer and other lawmakers have been working in recent years to improve the numbers of local police agencies participating in voluntary hate crime reporting efforts. Bills pending in Congress would give out grants to police forces to upgrade their computer systems; in exchange, the departments would begin uploading hate crime data to the FBI.

Beyer, who is sponsoring the House bill, titled the National Opposition to Hate, Assault, and Threats to Equality Act, said he would consider drafting new legislation to improve hate crimes reporting by federal agencies, or try to build such a provision into the appropriations bill.

“The federal government needs to lead by example. It’s not easy to ask local and state governments to submit their data if these 120 federal agencies aren’t even submitting hate crimes data to the database,” Beyer said.

In the Senate, Democrat Al Franken of Minnesota said the federal agencies need to do better. “I’ve long urged the FBI and the Department of Justice to improve the tracking and reporting of hate crimes by state and local law enforcement agencies,” Franken told ProPublica. “But in order to make sure we understand the full scope of the problem, the federal government must also do its part to ensure that we have accurate and trustworthy data.”

Virginia’s Barbara Comstock, a House Republican who authored a resolution in April urging the “Department of Justice (DOJ) and other federal agencies to work to improve the reporting of hate crimes,” did not respond to requests for comment.

ProPublica is a Pulitzer Prize-winning investigative newsroom. Sign up for their newsletter.