how to use from-to mode

[SB2020-eye]

Hi. I'm trying to use the "from-to" mode described in the README.md's "Generating images using generate.py." I figured the argument would be -f path/to/image1 path/to/image2 path/to/text.

The output is

usage: generate.py
      [-h]
      [-c CHECKPOINT]
      [-d SAVEDIR]
      [-g GPU]
      [-T]
      [-f CONFIG]
      [-a ADDTOCONFIG]
      [-r RUN]
      [-s STYLE_LOC]
generate.py: error: unrecognized arguments: path/to/image2 path/to/text

The list of potential arguments (above) seemed odd to me, since there is no "from-to" or "random" mentioned. So then I thought maybe I get prompted later in some way for these modes. So I ran it without the -f.

This returned

loaded iteration 60000
Traceback (most recent call last):
  File "generate.py", line 963, in <module>
    main(args.checkpoint, args.savedir, gpu=args.gpu,  config=args.config, addToConfig=addtoconfig, test =args.test,arguments=arguments, style_loc=args.style_loc)
  File "generate.py", line 252, in main
    num_char = config['model']['num_class']
KeyError: 'num_class'

I poked around in the code, especially looking for instances of num_char and num_class, but couldn't come up with anything enlightening. Any help would be much appreciated.

[herobd]

Sorry, run it with just the -c and -d (and possible -g) arguments and it will prompt you for the "mode" and then other paths.

[SB2020-eye]

Thanks, @herobd. That was what I was trying to describe in the 2nd half of my query above. I ran it just with -c and -d and -g arguments and got the 2nd code block you see above (with "KeyError").

[herobd]

Sorry I somehow totally missed the second part of your post. What checkpoint are you using?

[SB2020-eye]

I'm using: IAM_weights/IAM_auto_2tight_newCTC/checkpoint-iteration60000.pth. (But I'm not partial to it. I have no clue what the difference is between one and the next.)

[herobd]

That is the auto-encoder weights.
The weights for the generator are in IAMslant_noMask_charSpecSingleAppend_GANMedMT_autoAEMoPrcp2tightNewCTCUseGen_balB_hCF0.75_sMG

[herobd]

Sorry, I should have made the naming more obvious.

[SB2020-eye]

So I used the weights you pointed out above, and it just ran and ran--for 460 minutes until I just stopped it. No prompt, no readout (in my jupyter notebook), no output. Any idea what I'm doing wrong?

[herobd]

The directory specified with -d is empty?
Try generating another way with the "R" option (single image, also need -s to point to style pickle) to see if you can get it to produce something.
Also, providing the terminal output would be helpful.

[SB2020-eye]

Thanks. There was no terminal output, except something regarding me terminating it (^C...). And -d is definitely not empty.

Anyways, I have an update. I was setting -g to 0. I didn't realize 0 is assigned to my integrated graphics. So I set -g to 1, which points to my gpu card. Here is the terminal output I'm getting now:

Traceback (most recent call last):
  File "C:\Users\scott\Desktop\ManuscriptProject\Code\handwriting_line_generation\generate.py", line 962, in <module>
    with torch.cuda.device(args.gpu):
  File "c:\Users\scott\miniconda3\envs\hand\lib\site-packages\torch\cuda\__init__.py", line 289, in __enter__
    torch.cuda.set_device(self.idx)
  File "c:\Users\scott\miniconda3\envs\hand\lib\site-packages\torch\cuda\__init__.py", line 326, in set_device
    torch._C._cuda_setDevice(device)
RuntimeError: CUDA error: invalid device ordinal
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

I'm brand new to using CUDA and gpu. I'm not at all sure what's going wrong.

[SB2020-eye]

Okay. Sorry for the confusion. Two things.

1. -d IS empty. It's just an empty folder named 'output'. That is, the argument after -d is the path to an empty folder. (Is that what you mean by "empty"? And if so, it's supposed to be empty, right?)
2. There was actually no problem with GPU when I ran -g 0. I confirmed that this actually does point successfully to my external graphics card.

So I'm back to the original problem where it just runs and runs, no terminal output. I will try the "R" option, with -s next.

[SB2020-eye]

@herobd, you said:

Try generating another way with the "R" option (single image, also need -s to point to style pickle) to see if you can get it to produce something.

I thought you meant the -r parameter. But now I see you mean Random mode.

So...what I was trying to describe above is that when I run

!python generate.py -c C:/Users/scott/Desktop/ManuscriptProject/Code/models/IAM_weights/IAMslant_noMask_charSpecSingleAppend_GANMedMT_autoAEMoPrcp2tightNewCTCUseGen_balB_hCF0.75_sMG/checkpoint-iteration175000.pth -d C:/Users/scott/Desktop/ManuscriptProject/Code/output -g 0

nothing happens. It just keeps going until I ctrl+C. There is no prompt for mode. There is no terminal output. The cell just executes until I stop it (unless I should be waiting more than 460 minutes to see the prompt).

So, per your suggestion, I went to try the "R" option, but figured the same thing would happen (ie, that no prompt would appear and it would just keep running and running). Nevertheless, I ran

!python generate.py -c C:/Users/scott/Desktop/ManuscriptProject/Code/models/IAM_weights/IAMslant_noMask_charSpecSingleAppend_GANMedMT_autoAEMoPrcp2tightNewCTCUseGen_balB_hCF0.75_sMG/checkpoint-iteration175000.pth -d C:/Users/scott/Desktop/ManuscriptProject/Code/output -g 0 -s C:/Users/scott/Desktop/ManuscriptProject/Code/output/test_styles_175000.pkl

It did behave differently. After 4 seconds, I got the terminal output below:

loaded iteration 175000
Traceback (most recent call last):
  File "C:\Users\scott\Desktop\ManuscriptProject\Code\handwriting_line_generation\generate.py", line 963, in <module>
    main(args.checkpoint, args.savedir, gpu=args.gpu,  config=args.config, addToConfig=addtoconfig, test =args.test,arguments=arguments, style_loc=args.style_loc)
  File "C:\Users\scott\Desktop\ManuscriptProject\Code\handwriting_line_generation\generate.py", line 219, in main
    assert( len(all_style_files)>0)
AssertionError

[herobd]

Hmm. When you don't provide the styles, you might try putting in some print statements before generate.py line 260 (when it should prompt you for the mode) so see where it's spinning. Also, I'd recommeded running without GPU (no -g) to start with. It should get to the mode selection without any delay. Add -g back once you're able to start generating.

The error when adding the style pickle might be due to me not testing on Windows. I had it glob for the pickle files (I don't remember exactly why now) and I don't know if Windows does globs the same way. You should be able to take the glob stuff out (216-218) and just set all_style_files=[style_loc]

[SB2020-eye]

Thanks for the feedback.

I put a bunch of print statements in generate.py. None printed out. I even put one on line 1, and it didn't print anything.

[herobd]

You were getting other errors from the program. Is something capturing the std out?

[SB2020-eye]

Sorry. I'm not familiar with that and googling didn't help me. I'm not sure what you're asking. 😿 What should I be looking for?

[SB2020-eye]

Okay. Please see if this is correct. I put > stdout.txt at the end of my command line. Here is the content of the text file that was created:

loaded iteration 175000
indexes/random interp/vae random/strech/author display/math/turk gen/from-to/umap-images/Random styles/help/quit?

[herobd]

You need stdout to display in your terminal to interact with the script. The text you see in stdout.txt is prompting you to input the mode. I'm not sure why it isn't displaying in your terminal by default. I don't know Windows very well.

A work around could be to hard code your choices, replacing the inputs in the script.

[SB2020-eye]

Thanks. Your reply helped me to finally track down that my problem had to do with using a jupyter notebook. I was expecting the prompting to show in the code output block below the command. I don't understand it all, but it doesn't work that way in jupyter. So I ran from a normal terminal, and I finally got the prompt.

I chose from-to, then entered image path 1 & 2 and text to generate. Then the same prompt showed up again. What do I do at that point? And how do I see my results?

(I tried "quit." Then the prompt batch? appeared. I tried "y" and later "1", but both gave me errors.)

[herobd]

That should mean it generated the images. Be sure the directory specified by -d exists. You could also uncomment line 694 to be sure it's hitting the imwrite (image write)
https://github.com/herobd/handwriting_line_generation/blob/master/generate.py#L694

[SB2020-eye]

Sorry. I should have looked first. I just figured I was doing something wrong. The output was there. Thank you!

(What is the correct response to batch?)

[herobd]

I think that's the index in the data set to sample a style from.