It could be possible that you're simply overwriting your files with versions that don't have the tags in them. Double-check that your plugins and upload scripts don't have permission to overwrite these files without your permission.
If you are worried about your security though, the most common forms of injection of files is through image uploads and forums. Ensure that if you are using any database connections, that you use either MySQLi or PDO, and remember to use parameterised queries!
While you could make a few FTP or
.htaccess rules, they aren't really going to stop any potential hackers. The best thing you can do is to make sure you don't have any security vulnerabilities that can be exploited.
If you're running WordPress, run your site through WPScans to make sure there are no vulnerable plugins that you're using (there's literally thousands of vulnerable plugins).
For further reading on security vulnerabilities and how to address them, I recommend checking out the OWASP Top 10 cheat sheet.
Hope this helps! :)