You can delete an existing organization by clicking on the option Contacts from the More dropdown,

Within the contacts page, click on Organizations, find the organization that you want to delete and click on the Delete button next to that organization,

Deleting an organization will also remove it from all the leads where it was previously added.