By the end of the nineteenth century Middleton Hall had evolved into a complex of rooms and roofs that covered the whole of the current footprint of the Hall, as is shown by the early photographs above. The buildings that remain are all that were left after John Averill had reconfigured the Hall to his own requirements after buying the estate from Lord Middleton.
The worst of his attentions, which were to to open up the yard for his lorries, were the removal of the main entrance building and the damage done to the old domestic hall at the back of the yard, which he used as a garage. The oldest buildings were saved, including the original 13th century Hall, the Elizabethan Great Hall, a Tudor cottage and a later seventeenth century cottage, domestic hall and solar. The Georgian wing was used by the Averill family as their main residence.
What remains of the Hall was saved from falling down by a group associated with the Council for the Preservation of Rural England who between 1977 and 1980 negotiated with the then owners, Amey Aggregates, to set up a Trust to restore and preserve the property. Amey had bought the estate, extending to some 3600 acres, from Frederick (Dick) Averill, John's son, in 1966 to extends the gravel workings in the Tame valley. Middleton Pool was used to supply water to their operations, but Amey made no attempt to preserve the grade II* listed building, which was described in the Warwickshire Pevsner, published in 1966, as the longest continually occupied family home in Warwickshire.
The Lords Middleton had let the house to tenants from the end of the eighteenth century so that its best days were already behind it at that time. In the five hundred years before then it was the scene of some notable events and the home of some great people. These can be found under The People of Middleton.
As far as the building is concerned it started with Philip de Marmion, who was responsible for building the Norman Manor House, which is substantially in existence today.

The building is of stone and comprised a small hall and solar with an undercroft and a second room on the ground floor. The structure can still be seen in the Stone Building. It has a fine barrel vault roof as can be seen here.

After
Henry Willoughby inherited the property
it was extended by the addition of two long, thin wings and a Great Hall. The Great Hall remains
but the south wing today dates from the seventeenth century. It replaced the earlier structure
and contains a domestic hall and the solar, today called the Gallery, which is shown above.
The Jettied building is shown as found in 1977 on the left. It had been rendered to conform with the Georgian West Wing. On the right it is shown after restoration in 2002. The building is dated by dendrochronology to about 1530. The cottage behind the Jettied Building, inexplicably called the de Freville Building today, contained an inscription indicating that it was built ca 1647 probably in the same period as the south wing.
The Georgian West Wing was mainly built over the turn of the eighteenth century. The last part to be built was the south dining room, which replaced an earlier building in the tenancy of Sir Francis Lawley, who married a sister of the sixth Lord Middleton. The front of the Hall, in the picture on the right, shows the front of the West Wing and the outside of the Great Hall, the main structural features of which are of Tudor origin.