Drug delivery is a key component of nanomedicine, and traditional delivery relies on the adsorption or encapsulation of drug molecules onto nanomaterials. Many vehicles contain metal ions, such as metal organic frameworks, metal oxides, transition metal dichalcogenides, MXenes, and noble metal nanoparticles. These materials have high metal content and potential long-term toxicity issues, making clinical approval difficult. In this review, the latest progress in using drug molecules as ligands for metal coordination to form various nanomaterials and soft materials is summarized. In these cases, the ratio of drug to metal is much higher than traditional adsorption based strategies. Drug molecules are divided into small molecule drugs, nucleic acids, and proteins. The resulting hybrid materials mainly include nanoparticles and hydrogels, which can be grafted with target ligands to improve efficacy and further reduce toxicity. This article reviews the applications of these materials in addressing cancer, viral infections, bacterial infections, inflammatory bowel disease, and bone diseases. Finally, the future development directions were discussed from the aspects of basic research, materials science, medicine, etc.