A task can only be deleted until it has been synchronized with MS Project™.
Click with your right mouse button to open the context menu of the task you want to delete. Select the menu item Delete Task. After you confirm this action with OK the task is deleted. If there were attached work items you can find these work items at the end of the list.