You can remove contacts from a lead by clicking on the lead from the Pipeline view, 

Select the option Contacts from the left menu and click on the Remove button next to the contact that you want to remove,

Removing a contact from a lead will only delete the contact from that particular lead, it will neither affect the contact under Contacts nor any other lead where the same contact may be added.