Functional copolymer/organo-silicate [N,N'-dimethyldodecyl ammonium cation surface modified montmorillonite (MMT)] layered nanocomposites have been synthesized by interlamellar complex-radical copolymerization of pre-intercalated itaconic acid (IA)/organo-MMT complex IA-COOH center dot center dot center dot N(CH3)(2)-MMT as a "nanoreactor'' with n-butyl methacrylate (BMA) as an internal plasticization comonomer in the presence of radical initiator. Comparative analysis of physical structure, dynamic mechanical analysis parameters, and surface morphology of the obtained copolymers and their nanocomposites indicated that the interlayer H-bonding and flexible n-butyl ester linkages take place an important role in interlamellar copolymerization and intercalation/exfoliation of copolymer chains. It was found that nanocomposites' dynamic mechanical properties strongly depended on the force of interfacial H-bonding and amount of BMA units. An increase in both of these parameters leads to enhanced intercalation and exfoliation in situ processes of copolymer chains and the formation of hybrid nanocomposites. Copyright (C) 2009 John Wiley & Sons, Ltd.