Is it possible to send an email 'in background' using GuiXT, i.e. without showing the email window first?

Sending a background email is not a built-in possibility of GuiXT. You need to implement an ABAP function module, an .exe program or a dll that you can call from GuiXT.

Concerning the ABAP function: I do not know of any BAPI that sends an email, but perhaps one exists. To implement the function module on your own, see the SAP standard program BCS_EXAMPLE_1 for an example. In your script you use the "Call" statement to call the function remotely.

Concerning an .exe program: it can be implemented in any technology. In VB.NET it is relatively easy; use
Imports System.Net.Mail
Dim Smtp As New SmtpClient(...)