Some public administration literature that focuses on public administration reform indulges in constructing grand narrative theories such as New Public Management (NPM), or New Governance (NG). The most recent such theory that has been gaining attention over the past decade is the Neo Weberian State (NWS). The content of the theory with regard to its practical implication when it comes to reform is still unsettled. However, one key assumption behind the NWS is that reforms should be handled with care, as they may undermine the very institutions that have brought Western societies to the levels of their development they are in now. NPM’s drive to increase efficiency, and NG’s – democracy from the point of view of NWS is impossible if reforms deconstruct institutions that ensure the protection of the public interest and rule of law. NWS’s critique of post-communist reform efforts in the new eastern EU member-states is a case in point suggesting that leapfrogging the construction of a professional bureaucracy is ill-advised and does not lead to politically desired outcomes. In this paper we aim to evaluate which path of reform may lead to the best outcomes in a particular area of the public service – two non-military uniformed services of Lithuania – customs and penitentiary. These services have to a large degree avoided sweeping reforms throughout the independence period, in both cases – a soviet institutional legacy is also a factor. Both Customs and the Penitentiary service are suffering from multiple corruption scandals and very low public trust levels. These services are continuously in the crosshairs of reform, but few have been clearly articulated, even less so – implemented. Applying NPM to uniformed services, due to the nature of their functions was complicated, so as reforms in the 1990s and 2000s went along in the other sectors, policy-makers have largely left uniformed to their own devices, and this has led to retrenchment of the bureaucratic principles as would historical institutionalisms theory predict. We suggest that NPM-oriented governance avoids intervention in areas where NPM principles are hard to apply, leaving such areas without proper attention to continue down undesirable development paths. When the deconstruction of a hierarchy appears impossible, higher order governance needs to remain modelled as hierarchy as well. NWS in this case does offer a management modernization agenda, which could keep state institutions in step with social, technological, and economic developments.