It was originally called Fort Texas, built by General Zachary Taylor and the United States to establish the Rio Grande as the southern boundary of Texas, with walls more than nine feet high surrounded by a ditch fifteen feet deep and twenty feet wide.
After the death of Major Jacob Brown during the Mexican American War, it was renamed after the fallen hero. When the war ended, the fort fell into the hands of Juan Cortina at the time when he raided tried to occupy the City of Brownsville. Shortly after, Robert E. Lee was stationed at the fort. Eventually, with the approach of Union forces, Confederates set fire to the structure in 1863. During the Reconstruction period, the fort was reborn to what we see today.
The fort survived a yellow fever epidemic in the 1880s, the Brownsville Raid of 1906, and served as the headquarters for the Twelfth Cavalry from World War I to 1941. Seven years later, it was deeded to the city with the old fort hospital granted to Texas Southmost College, where it is still in use by the institution today.
Three areas that were once part of Fort Brown were designated a National Historic Landmark District in 1960.