It isn't mentioned in the article, but I think Johnson also became a slave trader... in that he brought the Africans over and sold them in the slave markets.
I went to a lecture years ago, regarding the early history of Knoxville. This was back in 84 or 85. The lecturer had written a book, and he talked about Johnson.
Prior to the Civil War, majority of blacks in Knoxville and surrounding area, were free. When things got tense just prior to Civil War, many left the area and went north. Many more left at the onset of the War, as they feared for their freedom and safety. Because of this, Knoxville's black population is about 6%, quite low for a "southern city" of it's size.