You can do this by selecting the audio block then, press the backspace key on your keyboard.