GIMP now has UI tests

(gimp.org)

36 points | by qtqtqt 1 hour ago

6 comments

  • pnkzin 4 minutes ago
    [dead]
  • sinoziqi 28 minutes ago
    [flagged]
  • dllu 8 minutes ago
    Funny how GIMP spawned the Gimp Toolkit, aka GTK, which is widely used in Linux, and yet GIMP's own UI seems to have long languished as a rather outdated design. Glad that they have tests, though, and hope they continue to improve it.
  • joegibbs 18 minutes ago
    They made a blog post to announce that they’d written thirteen unit tests? Sure, there’s value in handmade software and all that but GIMP is not really a particularly excellent piece of software - if anything was to benefit from unleashing the agents it would be GIMP.
    • Jolter 8 minutes ago
      I’d go so far as to say that in general, adding tests for legacy software before a refactor or rewrite is the perfect time to put the agent to work.
    • qtqtqt 2 minutes ago
      They must have set up a reusable harness, and it's the most important part.
    • altmanaltman 2 minutes ago
      The blog article doesn't say they have written thirteen unit tests.

      > *GIMP already has unit tests*. A unit test checks one small part of the code, for example how GIMP reads an XCF file. But our unit tests were not opening a real window. So it coult not tell us if a dialog opens, or if a menu item did its job.

      > Note that GIMP *already had tests of GUI code*, usually interacting directly with the core code, and sometimes even simulating input devices’ events (pointer movements, clicks, drags…). But nothing can beat interacting with the real interface. Furthermore such legacy tests directly hijacking core code come with their own problems, such as breaking all too often (and not because they detected a bug), all while not being able to detect whole categories of bugs.

      > UI tests check the program as a whole, in the same way as you use it.

    • WD-42 2 minutes ago
      Did you read the article? These aren’t unit tests. They are UI tests which are pretty hard to do well with desktop apps.
      • qtqtqt 0 minutes ago
        Especially given it isn't as easy as using Cypress.
    • hulitu 4 minutes ago
      > if anything was to benefit from unleashing the agents

      AI can write tests now ? For its own software ? Can it evaluate the results ? Correctly ? /s

  • rafram 47 minutes ago
    > We have 13 tests at this time.

    Does GIMP have a very restrictive AI policy or something? I get not wanting to ship a gigantic pile of UI tests without thorough review, but basically any model released in the last six months would be able to write UI tests covering every single control in an afternoon.

    • deskglass 37 minutes ago
    • samuria 38 minutes ago
      It’s frustrating to create tests with models, not sure if it’s everyone’s experience, but every time I asked it to create tests, it makes the tests conform to the sometimes broken code. Rather than calling out what the logic/code is wrong, it will make the tests conform to whatever is already broken.
      • forgotmypw17 33 minutes ago
        I recommend iterating on your prompts.

        Prod your agent first until it can explain back to you what's wrong with the tests.

        Then ask your agent to write a better prompt for writing tests (and save it to an .md file)

        Then reference that .md file and iterate on the tests again. It may take a couple of tries, but I've had good results with this strategy.

        Ended up with a whole prompting framework.

        • skydhash 1 minute ago
          Prodding an agent vs doing interesting things with my time? What should I choose?

          To be fair, some people like to prod. But, for me, I'd rather take hours reading manuals than do trial and error prodding for 30 minutes (even if I knew it would be that short). Yes it's not optimal in terms of time, but it's better for my soul and well being.

      • Waterluvian 35 minutes ago
        It can’t magically know expectations. If the code is well documented or there’s decent associated documentation, I’ve found it has no issue writing tests.
    • locusofself 20 minutes ago
      Not using AI to at least writes tests in this day and age is crazy work. I started a desktop application 2 months ago and it has almost 3,000 tests now, MANY of them UI tests.
      • timcambrant 6 minutes ago
        AI is a bit too excellent at writing tests. The number of tests doesn't really convey any information on test coverage. At least I've found that agents that naively improve codebases will write tests for pretty much anything, including irrelevant behavior that has little value in ensuring stability, security or correct behavior. It would probably help to run a test review stage with a strong model, where agents attempt to correct and consolidate tests to guard behavior rather than ensuring that theoretical code paths are followed.
    • hulitu 1 minute ago
      > any model released in the last six months would be able to write UI tests covering every single control in an afternoon.

      Microsoft also seems to use "models" to "test" its programs. Look how shitty they are.

  • akersten 17 minutes ago
    [flagged]