The Internet of Nano-Things (IoNT) is an emerging paradigm in which devices sized to the nanoscale (nanonodes) and transmitting in the terahertz (THz) band can become decisive actors in future medical applications. Flow-guided nanonetworks are well-known THz networks aimed at deploying the IoNT inside the human body, among other issues. In these networks, nanonodes flowing through the bloodstream monitor-sensitive biological/physical parameters and dispatch these data via electromagnetic (EM) waves to a nanorouter implanted in human tissue, which operates as a gateway to external Internet connectivity devices. Under these premises, two shortcomings arise. First, the use of the THz band greatly limits the nanonode’s communication range. Second, the nanonodes lack resources for processing, memory, and batteries. To minimize the impact of these concerns in EM nanocommunications, a novel dynamic multihop routing scheme is proposed to model in-body, flow-guided nanonetwork architecture. To this end, a reinforcement learning-based framework is conceived, combining the features of EM nanocommunications and hemodynamics or fluid dynamics applied to the bloodstream. A generic Markov decision process (MDP) approach is derived to maximize the throughput metric, analytically modeling: 1) the movement of the nanonodes in the bloodstream as laminar flow; 2) energy consumption (including energy-harvesting issues); and 3) prioritized events. A thoroughly THz flow-guided nanonetwork case of study is also defined. Under the umbrella of this case, diverse testbeds are planned to create a procedure of evaluation, validation, and discussion. Results reveal that multihop scenarios obtain better performance than direct nanonode-nanorouter communication, specifically, the two-hop scenario, which, for instance, quadrupled the throughput in a hand vein without sharply penalizing other aspects such as energy consumption.