Please watch this video, here we show how the attached files can be deleted: https://www.screencast.com/t/EWxtcqxgM

I don't understand why it's designed to not allow adding or removing attachments.

As I've already mentioned, removing is possible. The changing media files will be implemented in future versions. 

It's allowed only to delete the image, but not change it.  

Unless I'm missing something, there is no way to add or remove attachments when editing, or do anything at all with attachment. The reply dialog does not show the attachment(s) or any tools to manage attachments.

First screenshot I posted a comment with an attachment.

Second screenshot is editing that comment. 

On a related note, I don't understand why it's designed to not allow adding or removing attachments. It's not uncommon for someone to upload the wrong file or forget one file. Of course, they could just post a new comment or reply, but it seems they should be able to add or remove attachments when editing. 

This is designed to work in this way. It's not allowed to change the image if the comment is submitted. It's allowed only to delete the image, but not change it.  

Sorry, I don't have a link to test it.

Could you please leave an example URL to allow us to check the issue? 

Any media already uploaded is broken after saving an edit.