Historic Trails of Western NC
Smith-McDowell House: The People
Just a few miles from George Vanderbilt’s grand Biltmore Estate is Buck House, a different kind of mansion – one that was nearly 50 years old when Vanderbilt began construction. Saddled on a ridge between two hills just south of Asheville, ringed by picturesque mountains and overlooking the confluence of two great rivers, Buck House was constructed around 1840, over twenty years before the Civil War, most likely using enslaved laborers. During a time when most people lived in wood frame houses, the imposing structure was composed of brick, a rarity in early 19th century Asheville. Today it is known as the Smith-McDowell House, the oldest surviving house in Asheville and the oldest brick structure in Buncombe County, North Carolina.
The residents of the Smith-McDowell House were woven in to the fabric of Asheville. This driving tour begins at the Smith-McDowell House Museum and continues through historic sites related to early occupants of the house.
Please respect private property and parking restrictions.
01 / 07
1
Smith-McDowell House Museum
GPS Address: 283 Victoria Road, Asheville
Parking: Turn onto the gravel drive off Victoria Road. There is parking in the gravel lot beside the house. The grounds are open 24/7. Please check wnchistory.org for operating hours of our house museum.
Summary: This home was built c.1840 for the family of James McConnell Smith, a prosperous Asheville businessman, who is thought to be the first white child born west of the Blue Ridge Mountains. He was the son of Capt. Daniel Smith, an early settler who was involved in the forcible removal of native populations.
Begin your tour with a walk through the house and grounds to learn more about the people who walked here before you.
Image: Smith-McDowell House, ca1875.
2
Newton Academy Cemetery
GPS Address: Unadilla Avenue, Asheville
Parking: Turn on to Unadilla Avenue from Biltmore Avenue. Asheville Radiology is on the left. Veer right and the entrance to Newton Academy Cemetery is a few hundred yards up the road on your left. Park on the street.
Summary: Here you can visit the graves of Daniel and Mary Smith, and their son James, and his wife, Mary "Polly" Patton Smith. Though the family was originally buried in a family cemetery on the nearby hillside where Fernihurst Mansion now stands (70 Fernihurst Dr, Asheville, NC 28801), their graves were relocated to Newton Academy Cemetery in 1875 to make way for the new construction.
From the News: October 9, 1960, Asheville Citizen-Times - In the old Newton Academy Cemetery adjoining Newton Elementary School there stands a new tombstone marking the grave of the first child of white parentage born in North Carolina west of the Alleghanies [sic]. The marker was erected last week at the grave of James McConnell Smith, born June 24, 1787, within the present boundaries of the City of Asheville. The new marker, and the new markers at the site of other graves, were [sic] placed in the old cemetery by former U.S. Senator Robert R. Reynolds of Biltmore Forest, a great grandson of James M. Smith. James M. Smith was the son of Col. Daniel Smith, a native of New Jersey, who, after serving in the Indian wars and as an American soldier in the Revolutionary War, settled in Buncombe County. James Smith, whose mother was Mary Davidson Smith, married Polly Patton (1794-1853), daughter of Colonel John Patton, and settled in Asheville. He built and kept the Buck Hotel, where the Langren Hotel now is, operated a store on the opposite side of the street, maintained a tanyard at the present junction of Southside and Coxe Avenues, ran several farms, and built and for several years managed Smith’s Bridge, the first bridge in Buncombe County across the French Broad River. Smith later sold the bridge to the county. As the years went on, Smith became a large land owner in Asheville, other parts of the county, and even in Georgia. He died May 18, 1856, a wealthy man, and was buried on an estate called Fernihurst (now Viewmont) on Victoria Road. About 1875, his grave was removed to the Newton Academy Cemetery. The inscription on the old tombstone was becoming illegible, but can be read as follows: “He was the first child of white parentage born west of the Alleghany, in the present state of North Carolina and his course of life exhibited many qualities worthy of imitation by all those who come after him. He was a pattern of industry, frugality, energy and enterprise, a useful citizen, a warm friend and an honest man.” The inscription has been transferred to the new marker. The other new markers are for the nearby graves of Col. Smith, his wife, and the wife of James M. Smith.
About the Cemetery: Newton Academy Cemetery was established c1818 at the corner of what is today Biltmore Avenue and Unadilla Avenue. It is the final resting place for many of Asheville's early white settlers. Extensive research and mapping of the grounds was completed in the early 1990s by Dr. C. Michael Baker. His work revealed some 210 marked graves and approximately 85 unmarked graves. There are also memorial markers for both Union and Confederate Civil War soldiers whose bodies were not interred. Early graves included those of James McConnell Smith (the first owner of the Smith-McDowell House and, reportedly, the first child of white parents born west of the Blue Ridge Mountains); George Swain, the father of North Carolina governor David L. Swain; and stones marking the graves of many familiar Asheville names such as Alexander, Stevens, and Patton.
Most recent burial: 1913
Size: 2 acres
Number of burials: 210 marked graves, 85 unmarked graves
Notable grave sites: Daniel Smith, James McConnell Smith, Mary "Polly" Patton Smith, John E. Patton, George Swain
Administered by the Community Foundation of Western North Carolina
Image: Graves of James and Polly Smith in the Newton Academy Cemetery, May 29, 2020.

3
South Asheville Cemetery
GPS Address: 20 Dalton Street, Asheville
Parking: At the end of Dalton Street, Saint John "A" Baptist Church will be on your right. Park in the lot behind the church if the gate is not closed. The cemetery is not affiliated with the church, but is located beside it. As you stand facing the church, the cemetery is located to the left. The lower potion of the cemetery is next to the parking lot. Note that most of the graves in the cemetery do not have grave stones, but river rocks, as markers.
Summary: This cemetery is a 2-acre burial ground that serves as the final resting place for approximately 2,000 African Americans. Founded as a cemetery for enslaved people owned by the Smith and McDowell families, the first-known caretaker of this cemetery was George Avery (1844-1938), pictured at left. George Avery dug the graves and oversaw all the burials at the cemetery until his death in 1938. Any Black community member could be buried there for a small fee. Many were buried in wicker baskets or pine coffins. Graves were often only marked with field stones or handmade crosses. Of the nearly 2,000 people thought to be laid to rest there, only 93 have engraved headstones.
George Avery tracked all the burials by memory, so when he died in 1938, much of the knowledge of who is buried in the cemetery died with him. But why wouldn’t he leave written records, you ask? The answer should not surprise you.
In North Carolina it was illegal to teach an enslaved person to read or write beginning in 1818. By 1830, the law was amended to strengthen the penalty for doing so - a white person breaking the law would pay a monetary fine. A person of color breaking the law (whether free or enslaved) would “receive 39 lashes on his or her bare back.”
In land deeds signed by Avery as recently as 1922, when he would have been nearly 80 years old, he signs his name with an “X”.
About the Cemetery (from southashevillecemetery.net): The South Asheville Cemetery began as a slave burial ground, and its first known caretaker was an enslaved person named George Avery (1844-1938). Mr. Avery was owned by William Wallace McDowell (1823-1893), who lived in the Smith-McDowell House , and Mr. McDowell entrusted Mr. Avery as the manager of this cemetery, located on the family's property. Recognizing that the Confederacy was going to lose the Civil War, Mr. Avery decided to join the Union Army, and after the war Mr. Avery earned a pension as a result of his service in Company D, 40th United States Colored Troops. Upon his return to Buncombe County, Mr. Avery continued to oversee burials at the cemetery until his death in 1938, though he left no written burial records about the cemetery or its occupants. Mr. Avery's monument is one of only 93 headstones that have names or dates identifying the people buried at this site. The South Asheville Cemetery is a two-acre burial ground that serves as the final resting place for approximately 2,000 African Americans.
During the 20th century, the neighborhood surrounding the cemetery would come to be called South Asheville. This area was absorbed into Kenilworth and then, subsequently, into the City of Asheville. African American residents of South Asheville mostly attended two churches, St. John “A” Baptist and St. Mark A.M.E. Church. Over this same time period, the South Asheville Cemetery was one of only a few cemeteries for African Americans in the region, and it is notably the oldest public African American cemetery in western North Carolina.
Part of the South Asheville Cemetery was allotted for church congregants, but any African American community member could be buried in the cemetery for a nominal fee. Many of these people were buried in wicker baskets or pine coffins, their graves marked only by field stones or handmade crosses. Due to the settling of the ground and the array of unusual grave markers, the cemetery must be cleared by hand. The South Asheville Cemetery was closed after the City of Asheville annexed South Asheville and Kenilworth, and the last person interred there was Robert C. Watkins, buried in 1943.
The South Asheville Cemetery fell into disrepair during the mid-20th century, but in the 1980s members of the St. John "A" Baptist Church community--most notably George Gibson and George Taylor--began restoration efforts on the property. It was brought back to the public’s attention over this time period when a series of oral history recordings, now housed at the UNC-Asheville Special Collections Library, documented people’s recollections of the cemetery . Over the last 30 years, thousands of volunteers have worked with members of the South Asheville Cemetery Association to improve and maintain this sacred and historical site in an effort to promote greater public awareness of African American history in Buncombe County and to honor the people buried there.
Most recent burial: 1943
Size: 2 acres
Number of burial: 2,000+
Notable Graves: George Avery
Administered by the South Asheville Cemetery Association southashevillecemetery.net
Image: George Avery, at age 71 on February 18, 1917; Grave of George Avery in South Asheville Cemetery on May 29, 2020.
4
Buncombe County Courthouse
GPS Address: North Pack Square, Asheville
Summary: The current Buncombe County Court House is the 7th courthouse built in Asheville. In 1850, a Greek Revival-style brick courthouse (no. 5) was built - likely using enslaved laborers - approximately where the Vance Monument stands now. (It is important to note that Zebulon B. Vance, NC's Civil War governor, was also a large slaveholder.) It was on the steps of this courthouse that 13 people enslaved by the Smiths were sold in 1858. (The building burned in January 1865 and there are no known photographs.)
John Patton Smith (1823-1857) was the son of James McConnell Smith (1787-1856.) In the 1850 census James enslaved 44 people and when James died, his son John P. Smith inherited the Smith-McDowell House and 10 people. When John died a year later, at 34 years of age, without leaving a will, the 13 people he enslaved at the time were sold on the steps of the courthouse. It is likely that they were: Joe and his children - Alfred, Joe, Mary, Jane, and Vince - as well as Peter, Charles, Mose, Lucy, Harry, and, perhaps, Phillip.
Excerpt from James M. Smith’s Last Will and Testament, February 9, 1850: Seventh, I give and devise unto my son, John P. Smith his heirs and assigns forever, all my farm and land at the mouth of Swannanoa on both sides of French Broad River, which I purchased in part of Moses Smith, part from Daniel Smith, embracing the Col. Daniel Smith farm and the new brick house near the road, also one undivided half of my bridge and bridge tract of land on both sides of French Broad River purchased of John Jarrett to take effect at the death of my wife. Also, I give and bequeath to the said John P. Smith the following negroes Joe (the waggoner) and his wife Tilda* and her children, Alfred, Joe, Mary, Jane and Vince, also Peter, Charles (Sara’s son) and Robb* and their increase forever, also one half of the live stock…My old man, Phillip, has long been a faithful servant and useful to myself and my family, I direct no labor be required of him, but that he be allowed to live with my wife or my son, John P. Smith, as he prefers, and as the law requires that he must have an owner, I give him as the property of my son, John P., in confidence that he will take care of him and protect him, and I give and bequeath to Phillip twenty-five dollars per annum as long as he lives for his comfort, to be paid out of my estate. If the legacy cannot legally take effect directly to him, I leave it to be paid to John P. Smith having a confidence that he will faithfully apply it as here intended.” [Presumably, Phillip is the 66 year old man listed first on the 1850 slave schedule.]
*James M. Smith amended his will on January 7, 1854: “…whereas the boy Bobb [Robb] and the girl Tilda in my said will given to my son John P. Smith have since died, I will to him in their stead the boy Mose purchased at the widow Alexander’s sale, Lucy and Harry, the furniture and other property…”
Image: July 22, 1858, Asheville News
5
The Buck Hotel
GPS Address: 10 Broadway Street, Asheville
Summary: As a young man, James Smith “clerked for Mr. [James] Patton who owned the Eagle hostelry.” Smith married Mary “Polly” Patton (no relation to his employer) in 1814. In 1825, James Smith built and managed a third hotel -- the Buck Hotel -- that catered to the drovers moving animals to markets. The Smiths raised nine children on the premises. (This was prior to the construction of the Smith-McDowell House, c1840.) In addition to inn-keeping, Smith also ran a store across the street from the hotel and operated a tannery and a toll bridge across town.
After James and Polly died in the mid-1850s the hotel was leased to different managers and was known by a variety of different names. During the Civil War, the Buck Hotel was the official post office for the Confederacy in Asheville.
The Buck Hotel was torn down in 1907, to be replaced 3 times–by the Langren Hotel (1912), the BB&T parking deck (1964), and the A.C. Hotel by Marriott (2017), which is what you'll see as you drive by this corner today.
Fun Fact: When the hotel was torn down, the March 6, 1906 edition of the Asheville Citizen reported that "silver spoils of a counterfeiter's den was discovered yesterday morning when a portion of the old Buck Hotel was hurled down by crowbars." The coins, found hidden in a wall behind a chimney, individually wrapped in tissue, bore the date 1896.
From the News: March 4, 1906, The Charlotte Observer, Passing of Old Tavern. The Buck, Asheville's Oldest Hotel, to Give Place to Modern Six-Story Hostelry
The old Buck Hotel and the wooden structure adjoining are going. The work of tearing away these old landmarks was undertaken this morning. The work is in charge of McDowell & Spears and will consume probably two weeks. The buildings are located on North Main street in the very centre of Asheville and were built more than half a century ago. They are to be replace by a handsome and modern hotel of six stories of frame steel and concrete reinforcement with a roof-garden. The removal of the Buck Hotel means the passing of an ancient landmark; the taking away of the oldest structure in the city and a structure withal around which hovered much that had to do with Asheville when the city was a mere burg. The Buck Hotel at one time was a principal tavern in western North Carolina. It was for years conspicuous by a unique sign denoting its name--the antlers and head of a great buck. The removal of the Buck Hotel has been threatened for many years. It was some ten years ago that rumor had it the Buck was to go. This rumor inspired Will Aiken, now private secretary to the Governor of Montana, but then doing newspaper work in Asheville, to write what was declared to be the best story of the day. The story had to do with a "convention of bed bugs" held in the Buck Hotel "for the purpose of discussing the cruel report of destruction." But now the old landmark is assuredly going. Already portions of the buildings have been torn down and before the middle of the month all that will remain of the famous tavern will be piles of dust-covered timbers and soft-clay bricks. The tavern in the days before the war housed many prominent men. The Buck was the stopping place of hog and cattle drivers that passed through Asheville with great droves of animals before the days of the railroad. At that time there were great vacant stretches of land surrounding the tavern. This land was fenced and into these enclosures were fed great droves of hogs and cattle every night and day. It was a well-known stopping place for the drivers and usually these men spent the night at the tavern.
The building that will replace the old tumble-down frame structure will be a modern and commodious hotel. The hotel will be erected by C. H. Miller in charge of the Smith estate. It will be a thing of beauty and credit to Asheville. Plans for the building as accepted were drawn by Architect R. S. Smith of Asheville. The structure will be six stories in height with a frontage on North Main street of 128 feed and 129 feet on West College. It will cost about $125,000.
There will be 158 bedrooms, with telephone service in every room, two passenger elevators, a dining room to seat 200 persons, and 60 baths. There will also be an independent electric-light plant. One feature of the building will be a court with a glass roof which will extend from the main floor to the roof and the arrangements of rooms so that each room will be bounded by a corridor. On the sixth floor will be a ball-room, with a roof garden. Flowers and potted plants will find place on and around the roof garden and the whole structure will present an artistic appearance. There will be no bar-room in the building.
Image: c1890s, when the old Buck Hotel had become Mrs. Evans's Boarding House
6
Smith Bridge
GPS Address: Intersection Craven Street and Riverside Drive, Asheville
Summary: In the 1820s, James Smith invested in Buncombe Turnpike Company stock. At the same time, Smith also purchased prime land near the Turnpike and along the French Broad River. Drovers taking their livestock to market along the turnpike needed to cross the French Broad River safely, so James Smith purchased and operated the main ferry that crossed the river near Asheville and charged varying fees to each person based on type of vehicle and number of livestock. (He also began operating the nearby Buck Hotel to cater to these same customers.)
In 1834, Smith replaced the ferry with a wooden toll bridge (just south of the present-day Smokey Mountain Parkway Bridge on Patton Avenue). Because this was the only bridge across the French Broad, Smith made a lot of money. As examples, Smith charged $0.50 a loaded four-horse wagon, $0.25 for a loaded one-horse wagon, $0.02 per cow, and one half cent for each turkey.
When another bridge - which would be free to cross - was proposed in 1844, Smith unsuccessfully petitioned against it. However, Smith continued to collect tolls until he sold his bridge to the county and it, too, became public.
According to NCpedia: "The wooden bridge was replaced by an iron bridge in 1881. The iron bridge was replaced by a reinforced concrete bridge after the flood of 1916. That bridge was closed in the 1970s and was, finally, replaced in the 1990s. Today, this low bridge is adjacent to the Smokey Mountain Parkway Bridge on Patton Avenue and connects to West Asheville."
Image: Smith Bridge (center) in 1918
7
Riverside Cemetery
GPS Address: 53 Birch Street, Asheville
Summary: Sarah Lucinda McDowell was one of James and Polly Smith's daughters. She and her husband, William Wallace McDowell, purchased what is now known as the Smith-McDowell House in 1858 from the estate of her brother, James Patton Smith who died without a will.
The McDowells lived in the house for 25 years and raised a large family there. William Wallace McDowell died of a heart attack on June 22, 1893. Sarah Lucinda died of pneumonia at her home on South Main Street at 2 A.M. on November 1, 1905, the oldest citizen of Asheville at the time. Both she and her husband are buried at Riverside Cemetery as are other of the houses' later occupants, including: Robert Garrett, Alexandra Garrett, Robert Johnson, and Herman Gudger.
From the News: November 2, 1905, Asheville Citizen - Mrs. McDowell belonged to one of the oldest and most respected and influential families of Western North Carolina. She was the daughter of James M. Smith, the first white person born west of the Blue Ridge in what is now the state of North Carolina. Her mother was a daughter of John Patton, another pioneer in the transmontane country, and her relatives are numbered by the hundreds in this western section of the state…. Mrs. McDowell had been a member of the Presbyterian Church since her girlhood, and her friends, outside her large circle of relatives, were very numerous. She was a devoted mother, a sincere friend, and a kind and generous neighbor, and many hearts will be saddened by her death. During her declining years she had the tenderest of care at the hands of her children, the devotion beautiful as it was loving and constant.
She was buried next to her husband in Section E, Lots 1 and 3, Grave 5.
About the Cemetery: Riverside Cemetery is also the burial place of noted authors Thomas Wolfe and William Sidney Porter, better known as O. Henry. Other prominent citizens whose graves can be viewed here include: Isaac Dickson (the first African American individual to be appoint to the Asheville City School Board), Queen Maude Carson (Asheville's first female public school principal), George Masa (famous Japanese photographer), James H. Posey (bodyguard to Abraham Lincoln), Richard Sharp Smith (supervising architect for the Biltmore House, who also renovated the Smith-McDowell House), Lillian Exum Clement (the first female legislator in the South), Francesca Guastavino (wife of the noted architect, Rafael, who built arched tile domes across the US, including the Basilica of St. Lawrence in Asheville), and the remains of 18 German sailors from WWI.
Most recent burial: ongoing
Size: 87 acres
Number of burials: 13,000, 9,000+ markers, 12 family mausoleums
Administered by Asheville Parks and Recreation Department.
Image: William and Sarah McDowell, no date