In this paper, we present a novel artificial intelligence-based fog controller, called FogAI that provides a versatile control mechanism to the fog layer. FogAI not only abstracts the control mechanism from the fog environment but also offers potential solutions for the problems of fog-based Next Generation Internet of Things (NGIoT) systems. To this end, we first present a comprehensive examination of challenging issues in Fog Computing (FC). Then, we outline possible FogAI based solutions to these challenges from different perspectives. To illustrate the feasibility of our FogAI concept, we design a use case scenario for task offloading problem in FC. Then, we propose a Deep Q-Learning (DQL) algorithm that autonomously performs task offloading in delay-sensitive and computationally-intensive IoT applications and test it on FogAI. The results show that the proposed FogAI-assisted DQL algorithm is superior to existing offloading policies.