Capturing console output with Delphi 2010/XE (revised)
Following my previous post (Capturing console output with Delphi 2010/XE) and with all the great comments received on it, I have decided to publish the new solution provided by Lübbe Onken which solves the hanging issue when capturing the output for different kind of commands like ping, netstat, etc. The problem occurs on the last ReadFile which this solution will fix.
Here I'm summarizing Lübbe Onken comments on this issue:
"The current implementation assumes that if the external process is not finished yet, there must be something available to be read from the read pipe. This is not necessarily the case. If we use PeekNamedPipe to check for available data before entering the internal repeat loop, everything is fine.
I also put the CloseHandle into try ... finally and moved Application.ProcessMessages behind the internal read loop, because IMHO the screen update is better handled after processing the callback than before. But this is just cosmetic."
Here you can find the source code:
usage:
I want to say thanks to everyone who spend time looking at this issue and for making the community work and grow.
Jordi
Very interesting, thank you.
ReplyDeleteNow a related question: how to interrupt long tasks that are being captured from console?
i.e. say I want to interrupt traceroute www.google.com after the first few hops?
Hi Fabio,
DeleteJust add a boolean variable on the repeat to exit it:
until (dRunning <> WAIT_TIMEOUT) or breakLoop;
This will help you break the loop.
Jordi
Great!
ReplyDeleteThank you very much.
You're welcome!. Thanks for your comments.
DeletePretty cool.
ReplyDeleteThanks!
DeleteFinally I found working example. Thank you!
ReplyDeleteThanks!
DeleteThank you! Been looking for exactly this example. Tried a number of different console capture examples, some hanging as mentioned. None worked for use with the FireBird console app, GFIX. Works beautifully. For anyone else wanting to apply it to GFIX, just remember to disconnect from the database before running GFIX.
ReplyDeleteChuck Belanger
You're welcome Charles. Great that it is useful!!
DeleteJordi
that's really interesting, with a smart way!
ReplyDeleteI just receive this error:
"ERROR: The target system must be running a 32 bit OS."
(running XE on WIN7 x64)
...is there a way to solve it... or is it impossible?
Hi Filippo,
DeleteI can't get this error and I'm running it under Windows 7 64 bits as well.
What are you trying to execute in the command line?
Jordi
hola, supongamos que tengo que enviar un ctrl+c para cancelar la accion de algun programa, como lo podria hacer?
ReplyDeletesaludos
Hola,
DeleteSi miras en los comenarios defino un breakLoop. Puedes marcarlo como true si pulsas las teclas cntrl+c y tener asi mas control en tu aplicación.
Jordi
Thanks Jordi,
ReplyDeleteit's definitively an interesting way!
Does it functions under delphi 2007?
Hi,
DeleteIt should work, but you will have to test it.
The section that uses generics and anonymous methods won't work though.
Jordi
hey, im usin it for ffmpeg and after ~10 minutes it crashes. Any idea?
ReplyDeleteHi, Could it be the ffmpeg is doing something else behind the scene? I have used this approach for batch files running multiple applications lasting for 40 min and it works like a charm. Never had any issue.
DeleteDo you get any sort of error?
Post it here and we'll have a look.
Jordi
Hello,
ReplyDeleteWhat a shame I so not know how to use it. Should I trat it as a component or prepere component somehow or maybe I shoulf just add it to my code in unit1(form 1)? Sorry for that, I am starting, but It would solve all my problems.
Anja
Hi Anja,
DeleteJust put it in a form and run it.
Jordi
Thank you for help, but one more question. Does it communicate with console or just capture the output. I mean do I need to prepare any connection with the console? Why in usage there is 'java -version'?
DeleteAnja
Hi Anja,
DeleteThe methode creates a pipe with the console, so it captures everything is being dumped. You don't have to prepare anything, the method is sorting all those things for you. I'm using java -version because this command displays values in the console. Just type in your cmd -> java -version and you should get something similar to:
java version "1.7.0_05"
Java(TM) SE Runtime Environment (build 1.7.0_05-b05)
Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)
etc.
I hope this sorts your issues.
Jordi
I have already tried something like this. It is only a trying, cause I'd like to delete DosCommand in my original project and reprece it with this solution. Unfortunetelly nothing works. I do not know why, I am sure that I did something wrong. Could you take a look, please? It is a new project to text it(and also does not work):
ReplyDeleteSource code:
unit Unit1;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls;
type
TForm1 = class(TForm)
Memo1: TMemo;
Edit1: TEdit;
Button1: TButton;
procedure Button1Click(Sender: TObject);
procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
private
{ Private declarations }
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.dfm}
//Anonymous procedure approach by Lars Fosdal
type
TArg = reference to procedure(const Arg: T);
procedure TForm1.CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
const
CReadBuffer = 2400;
var
saSecurity: TSecurityAttributes;
hRead: THandle;
hWrite: THandle;
suiStartup: TStartupInfo;
piProcess: TProcessInformation;
pBuffer: array [0 .. CReadBuffer] of AnsiChar;
dBuffer: array [0 .. CReadBuffer] of AnsiChar;
dRead: DWORD;
dRunning: DWORD;
dAvailable: DWORD;
begin
saSecurity.nLength := SizeOf(TSecurityAttributes);
saSecurity.bInheritHandle := true;
saSecurity.lpSecurityDescriptor := nil;
if CreatePipe(hRead, hWrite, @saSecurity, 0) then
try
FillChar(suiStartup, SizeOf(TStartupInfo), #0);
suiStartup.cb := SizeOf(TStartupInfo);
suiStartup.hStdInput := hRead;
suiStartup.hStdOutput := hWrite;
suiStartup.hStdError := hWrite;
suiStartup.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;
suiStartup.wShowWindow := SW_HIDE;
if CreateProcess(nil, PChar(ACommand + ' ' + AParameters), @saSecurity, @saSecurity, true, NORMAL_PRIORITY_CLASS, nil, nil, suiStartup,
piProcess) then
try
repeat
dRunning := WaitForSingleObject(piProcess.hProcess, 100);
PeekNamedPipe(hRead, nil, 0, nil, @dAvailable, nil);
if (dAvailable > 0) then
repeat
dRead := 0;
ReadFile(hRead, pBuffer[0], CReadBuffer, dRead, nil);
pBuffer[dRead] := #0;
OemToCharA(pBuffer, dBuffer);
CallBack(dBuffer);
until (dRead < CReadBuffer);
Application.ProcessMessages;
until (dRunning <> WAIT_TIMEOUT);
finally
CloseHandle(piProcess.hProcess);
CloseHandle(piProcess.hThread);
end;
finally
CloseHandle(hRead);
CloseHandle(hWrite);
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
CaptureConsoleOutput('java -version', TEdit1.Text,
procedure(const Line: PAnsiChar)
begin
Memo1.Lines.Add(String(Line));
end
);
end;
end.
Hi Anja,
DeleteEverything looks OK. What version of delphi are you using? Check with other comments. Instead of typing java -version, try typing any other command you want to display in the memo.
Regards,
Jordi
Hello,
Deletefirst of all thank you for helping me. I am using delphi 2010. To be clear Embarcadero RAD Studio 2010. Previously I used Delphi 2007 and it is an environment I know much better. The problem as far as I can see is with two lines:
unit Unit1;
interface
uses
(...);
type
(...);
procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg); //here is a problem cause I have not declared TArg<>
When I try to declare it before the procedure itself I get information that ":" expected "<" found.
Im getting confused. If you have any idea I will be grateful..
Anja
Well,
DeleteThank you very much for help. I started program, there was only stupid mistake I repeat type formula 2 times. I will inform you about results of using it. Thank you very much.
Anja
Hi Anja,
DeleteGlad that you are sorting out your issues. Please let us know your results.
Jordi
Yes... This worked 1000%...
ReplyDeleteThank very much...
Yes... This is worked very well...
ReplyDeleteThank you so much...
Thanks to you for using it!
DeleteWow this is great, working on my TOR console capture on Delphi XE2, i have search in many time, many of those hang in form, but i found this usefull here
ReplyDeletethank 4 all,...
Thanks for reading it!
DeleteIt does not work for me, maybe someone has an advice for this.
ReplyDeleteI have Windows7 64 and DelphiXE5
First I tried java -version which makes the CreateProcess return false.
Then I tried putting 'CMD /c '+command in the CreateProcess, which makes CreateProcess return true, but the message is that it cannot find java.
I am trying this for quite some time and I run out of possible solutions.
I found what went wrong.
ReplyDeleteI was trying to start 64bit Java from a 32 bit application. This does not work.
Hi Bert,
DeleteGlad you found it!.
Jordi
Thank you very much!
ReplyDeleteThanks to you!
DeleteExcelente Jordi, buen trabajo.
ReplyDelete- Acabo de implementar para crear backups con rar.exe
- El aplicativo es disparado por medio de un method con DataSnap y es perfecto sin problemas.
Gracias.
Startkill
Lima-Perú
Perfecto!
DeleteHi, I am having trouble getting this to work. I am running DelphiXE. Any help you can offer most appreciated.
ReplyDeleteI have a form, with a memo, an edit box, and a button. I get these error messages:
[DCC Error] Unit1.pas(15): E2003 Undeclared identifier: 'TArg'
[DCC Error] Unit1.pas(34): E2037 Declaration of 'CaptureConsoleOutput' differs from previous declaration
[DCC Error] Unit1.pas(15): E2065 Unsatisfied forward or external declaration: 'TForm1.CaptureConsoleOutput'
[DCC Fatal Error] Project1.dpr(5): F2063 Could not compile used unit 'Unit1.pas'
Here is the full code:
unit RunCmdLine;
interface
uses
Windows, Messages, SysUtils, Variants, Classes, Graphics, Controls, Forms,
Dialogs, StdCtrls;
type
TForm1 = class(TForm)
Memo1: TMemo;
Edit1: TEdit;
Button1: TButton;
procedure Button1Click(Sender: TObject);
procedure CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
private
{ Private declarations }
public
{ Public declarations }
end;
var
Form1: TForm1;
implementation
{$R *.dfm}
//Anonymous procedure approach by Lars Fosdal
type
TArg = reference to procedure(const Arg: T);
procedure TForm1.CaptureConsoleOutput(const ACommand, AParameters: String; CallBack: TArg);
const
CReadBuffer = 2400;
var
saSecurity: TSecurityAttributes;
hRead: THandle;
hWrite: THandle;
suiStartup: TStartupInfo;
piProcess: TProcessInformation;
pBuffer: array [0 .. CReadBuffer] of AnsiChar;
dBuffer: array [0 .. CReadBuffer] of AnsiChar;
dRead: DWORD;
dRunning: DWORD;
dAvailable: DWORD;
begin
saSecurity.nLength := SizeOf(TSecurityAttributes);
saSecurity.bInheritHandle := true;
saSecurity.lpSecurityDescriptor := nil;
if CreatePipe(hRead, hWrite, @saSecurity, 0) then
try
FillChar(suiStartup, SizeOf(TStartupInfo), #0);
suiStartup.cb := SizeOf(TStartupInfo);
suiStartup.hStdInput := hRead;
suiStartup.hStdOutput := hWrite;
suiStartup.hStdError := hWrite;
suiStartup.dwFlags := STARTF_USESTDHANDLES or STARTF_USESHOWWINDOW;
suiStartup.wShowWindow := SW_HIDE;
if CreateProcess(nil, PChar(ACommand + ' ' + AParameters), @saSecurity, @saSecurity, true, NORMAL_PRIORITY_CLASS, nil, nil, suiStartup,
piProcess) then
try
repeat
dRunning := WaitForSingleObject(piProcess.hProcess, 100);
PeekNamedPipe(hRead, nil, 0, nil, @dAvailable, nil);
if (dAvailable > 0) then
repeat
dRead := 0;
ReadFile(hRead, pBuffer[0], CReadBuffer, dRead, nil);
pBuffer[dRead] := #0;
OemToCharA(pBuffer, dBuffer);
CallBack(dBuffer);
until (dRead < CReadBuffer);
Application.ProcessMessages;
until (dRunning <> WAIT_TIMEOUT);
finally
CloseHandle(piProcess.hProcess);
CloseHandle(piProcess.hThread);
end;
finally
CloseHandle(hRead);
CloseHandle(hWrite);
end;
end;
procedure TForm1.Button1Click(Sender: TObject);
begin
CaptureConsoleOutput('java -version', Edit1.Text,
procedure(const Line: PAnsiChar)
begin
Memo1.Lines.Add(String(Line));
end
);
end;
end.
Thanks,
Aaron
Hi Aaron,
DeleteYou need to move the declaration of TArg on the first apparition of type.
Regards,
Jordi
HI Jordi,
DeleteI've stuck with this problem. you told that
" You need to move the declaration of TArg on the first apparition of type "
So, where would i declare of it.
Thx very much.
Jordi
Hi Jordi,
ReplyDeleteThis procedure is working if a issue a 'cmd /c dir /s c:\*'
But i don't have success using it to display a 7za.exe (7-zip command line) output. The pipe doesn't not return line by line progress.
I mind it is related to 7za.exe but can't find what I can do.
Any ideas?
Thanks.
João
Hi João,
DeleteThat's because 7za.exe must display the internal results differently and not in the pipe.
Regards,
Jordi
Hi Jordi,
DeleteCould you please try following command:
7z.exe x -y "C:\Linux.iso" -o"E:\"
7z.exe is changing the last line of the console output like %1, %2, %3 and so.
But the %1, %2 lines does not come to the pipe.
Is there any way to capture the %1, %2 output?
I am looking for a solution for days.
Thank you.
Hi Jordi,
DeleteCould you please try the following command with Windows Dos Command Prompt and with the Delphi CaptureConsoleOutput procedure:
7z.exe x -y "C:\Linux.iso" -o"E:\"
7z.exe is changing the last line of the output with %1, %2, %3, ... and so. It is not adding new lines. It seems changing the last line.
We can't capture the %1, %2, %3 output with pipe (CaptureConsoleOutput procedure).
Is there any way to capture %1, %2 output?
I am looing for a solution for days. How can I capture it?
Thank you.
A better solution is to close your handle to the write end of the pipe (hWrite) after calling CreateProcess. That way, ReadFile() will exit with ERROR_BROKEN_PIPE when the child process exits, and you don't have to poll.
ReplyDeleteThis a best solution!
ReplyDeleteCan you help me about words with accent or special character return? Because in my return like a SVN command the files with word accented return String is broken.
Hi Jordi
ReplyDeleteI have a console application that uses Windows' SetConsoleCursorPosition to output text at some particular position. For example it writes on the console: 'time step is 0.04 sec'. The user then sees the 0.04 updating each time step, on the same line in the console window. Otherwise the window would have thousands of lines.
Now when I use the CaptureConsoleOutput function the memo grows to thousands of lines, because the call back function adds lines to the memo.
Do you think it is possible to change this so that the output is similar to the original console ouput, i.e. that it updates certain positions in the memo instead of adding lines?
regards,
Jan
Hi Jan,
DeleteYou'll have to play with it. Probably you could buffer it previously and then render the results later on.
Regards,
Jordi
This works in Delphi XE7, but NOT IN REAL-TIME. The output is written to the memo only when the console program terminates. So how can I make it work in real-time?
ReplyDeleteHi Peter,
DeleteThe pipe is opened and awaits the finalisation of the process. That's why WaitForSingleObject is used for so then the console output can be captured. To do what you are expecting, you would have to look for an alternative way.
Regards,
Jordi
Hi,
ReplyDeletemany thanks for this wonderful help. Is it possible that you write a version that does the output only after executing not while executing? All examples I found do not work, maybe you can help?
Thank Jordi. This is the best and perfect for me exactly. I am always using it when need to capture the console output in real time.
ReplyDeleteOne most important question for me, how can i ABORT or CANCEL a process which take a long process?
An ilustration, i put a TMemo, TEdit and TButton.
i'am running FFMPEG command line to convert a big size video file into other format. for example: ffmpeg -i video.mp4 -vn -ab 256 audio.mp3
This process could sometime need more than 30 minutes. SO i want to abort it before it's finished without closing application. How to do this?
while in native CMD window i just can do it with a simple task by sending CTRL+C keystroke to the console. Please help.
Btw, i am using it in a simple thread execute like this:
TConvert = class(TThread)
protected
procedure Execute; Override
end;
No matter it's running in thread or not, i just need to be able to Cancel it when needed.
Again, please help. thank you
Hi There,
DeleteProbably you'll have to break the look. I think that if you look in the comments you might find the solution. I do remember this:
"Just add a boolean variable on the repeat to exit it:
until (dRunning <> WAIT_TIMEOUT) or breakLoop;" So have a look as to see if this makes sense to you.
I know it was a bit tricky to stop the pipe but I think it's doable.
Cheers,
Jordi
Thank you Jordi. I'll try to dig it more
DeleteThis comment has been removed by the author.
DeleteHi! I have some issues with this. It breaks some of the sentences, as well as it being printed in "chunks" rather than one line at the time.
ReplyDeleteI have checked manually directly from the cmd, and there it does not have the same print.
I have tried to change the CReadBuffer, and it seems to change the behaviour, but still not able to make it proper.
Can you give some advice?
I have been working on this for days now... no one?
DeleteIt is important that my delphi app shows exactly the same as the cmd window. I've made sure there is nothing in my code making it to break the line(sentance). It's clear that it breaks when the buffer is filled. This process only prints after certain buffer is reached, so you dont need to wait for the whole process is finished. Thats good if you only need it printing and for people to read and understand it. However, if your program are using the output for errorhandling etc, it will not work well.
Late to the party but try changing
DeleteMemo1.Lines.Add(String(Line));
to
Memo1.SelText := String(Line);
Hi, Jordi.
ReplyDeleteI need to use that Procedure on my Delphi 7, is that possible ?
please help. Thank You in Advance