Fortunately this is a pretty simple issue to take care of.
Go ahead and craft yourself another computer, a disk drive, and a floppy disk. Set the disk drive down next to the newly crafted computer and insert the floppy disk into the disk drive. Start up the computer and type "edit disk/startup". It will open the editor: at this point just save and close the file – you do not need any content in the startup file. Pick up the disk drive and floppy and move it to the computer that is stuck in a loop. Once you've put the floppy disk in, you can start your computer and the computer startup code will be bypassed in favor of the blank startup file on the floppy disk.
However, I should note that if you cannot terminate the program, it is probably due to a lack of "sleeping" in your code. For example, say you have a while loop like this:
while true do
print(1+1)
end
This code never sleeps, and is therefore very difficult to terminate (ComputerCraft is supposed to have protections against this sort of thing, but I've found that on slower computers they do not work as well).
In order to help with the termination issue, I usually add a simple sleep(0.05) statement at the bottom or top of my loops:
while true do
print(1+1)
sleep(0.05)
end
Now the code should be easily terminated and you won't have to deal with floppy disks.