A noble band of black and white members worshiped in Providence Baptist Church, Milford, Virginia. In the year 1867, the Negro members, led by Patrick Lewis, Seth Boswell, Samuel Shepherd, Spencer Sayles, Lewis Boswell, James Carter and Joseph Armstrong organized their own church. This structure was a brush arbor located on the road from Bowling Green to Sparta. It was named Mt. Barron. The pastor was Rev. Moses Johnson. A clerk was hired from Mt. Zion by the name of John Miller. It is believed that the church was allowed to use the land at no charge. That structure was destroyed by fire. Another building was constructed across the road (location of the 1980 structure) from where the original structure stood. Historical data on this is unclear but it appears that this second structure may have been a log home. This structure was named Mt. Evans. The leader was Rev. Elmore Taylor. They prospered until this building, too, was burned. In the year 1880, an acre of land was purchased by the trustees of Mt. Evans Baptist Church from Julia A. Gatewood, Mariah B. Gatewood and Margaret E. Chapman for the sum of $20.00. The deed was made on October 15, 1879. The acre was bounded by the lands of Richard Green, Esq. and the main road leading from Bowling Green to Sparta. The transaction was recorded in the Clerk's Office on June 5, 1880. William Brookes, trustee, received Mt. Evans' copy of the deed. A new structure was then built and preserved until 1980.
Rev. A. L. Goodloe, the pastor suggested changing the name from Mt. Evans to St. James. He presented a wooden emblem in the shape of a half moon. The moon represented peace, humility and serenity. Inscribed upon the emblem were the words "St. James Baptist Church - Built in 1880.".