To remove a module cleanly, first uninstall it through the admin interface at Extend then Uninstall, or via Drush with drush pm:uninstall modulename. This removes any database tables, configuration, and data the module created.
Only after uninstalling should you remove the code itself with composer remove drupal/modulename. Removing the code first can leave orphaned data and cause errors.
Some modules cannot be uninstalled while content depends on them. For example, a field type module refuses to uninstall while fields of that type exist. In those cases you need to remove the dependent content or configuration first. Always test module removal on staging before production, since some modules do more cleanup than others.