The with-x style macros can indeed be replaced by functions that take a closure. E.g. in Scheme you have the function (call-with-input-file string proc).

The macro is slightly more concise. Contrast

  (with-input-file (x "x") (print (read x))) ; Common Lisp

with

  (call-with-input-file "x" (lambda (x) (print (read x)))) ; Scheme

No runtime consing of closures either! In fact, in Common Lisp there is a style of writing call-with-foo that takes a closure and wrapping that with a macro with-foo. Since the macro abstracts that detail of having call-with, it is much more flexible. For example the macro could expand some stuff known at compile time and do some compile time computation too., instead of the closure consing function.

I feel pg314 did an adequate job answering your first question -- although I would have explained it that any meta-programming macros must be, by definition, higher-order functions. The only differences are giving the consumer function inline syntax and partial execution at compile time: both potent benefits.

With-file might not be the very best example of "what if you forget", because it's so very simple. A more complex example -- where "forgetting" might be more than just a single function call -- would be handling semaphores and locks for threading. A well-written "with-lock" macro could ensure that these resources are obtained and released in a determined order (i.e. compute some sort of hierarchy for locks and enforce this when obtaining multiple locks).