The fortress in Srebrna Góra was built in 1764–1777 by the order of Frederick II, the King of Prussia. It was designed by Prussian architect Ludwig Wilhelm Regeler, aided by a number of Prussian military engineers. Minor additional works took place in the following years, but no major alterations were made; construction of a nearby flanked fort was begun but was quickly abandoned. The complex is composed of six forts, several bastions, and associated elements. The main fort of the complex is the central Donżon Fort on the Warowna Góra hill.
The complex is located on the heights of the Sudety Mountains, a body which forms a natural border between the Kłodzko Valley and the Silesian Lowlands. It controls the passage through the Silver Valley (Polish: Przełęcz Srebrna, German: Silberbergpass). The fort spans three hills: Ostróg (627 meters above sea level), Warowna Góra (686m), and Wielki Chochoł (740m). The fort could shelter a garrison of 4000 soldiers, supplied to survive a year-long siege. It was defended by 264 artillery pieces. The fortress was intended to guard a route linking Prussian territories with Bohemian lands in the south, and thus help repel any possible incursions from Austria. The cost of the construction was reported as 4.5 millions of Prussian thalers.
The fortress was never captured by the enemy while besieged. On 28 June 1807 it successfully resisted a siege by Napoleonic forces during the War of the Fourth Coalition; this was the only time the fortress was the site of an active battle. By 1860 it was declared obsolete and the garrison reduced in size; it was abandoned as an active military stronghold in 1867. The fortress has survived till modern day with relatively little modernization or damage, contributing to its valuable status as a historical monument of its era. It served as a military training grounds, and by the end of the 19th century was already a tourist attraction, with a restaurant opened in the fort by 1885. A youth hostel opened there in 1913, a museum in 1931, and by the 1930s it was visited by 50,000 tourists each year.
During World War II the fortress served as Prisoner-of-war camp (Oflag VII „b” 1939-1941, Stalag 367 1941–1945). It was used as a prison for Polish officers imprisoned by the Germans. Notable prisoners included Counter Admiral Stefan Frankowski, General Tadeusz Piskor and Rear-Admiral Józef Unrug. After the war, the fortress was parts of the lands transferred from Germany to Poland. In 1961 it was added to the Polish register of objects of cultural heritage. In 1973 a military museum was opened in the Fort. In 2002 it was declared a culture park, and in 2004 it was recognized as a Historic Monument.