You can delete a todo task from the lead details page. 

To delete a todo item, click on the option Todos from the lead details page.

In the Todos view, click on the trashcan icon next to the todo item.

The deleted todo item will also be removed from the My Todo List of the user assigned to the lead.