It writes more informative commits than I could ever make so I’m just reading what it says and mostly copy/pasting completely most of the time, I write all of the changes I’ve made into an LLM with a large context window and it write a very detailed commit not just with a title but with bullet points describing each of the changes precisely

  • Solemarc@lemmy.world
    link
    fedilink
    arrow-up
    10
    ·
    edit-2
    11 months ago

    Why do you need to spend a “considerable amount of time crafting commit messages?” That feeding your code into an LLM and getting it to summarise for you is faster? I don’t understand how this could possibly streamline anyone’s workflow? What do your commit messages normally look like?

  • TootSweet@lemmy.world
    link
    fedilink
    English
    arrow-up
    9
    ·
    edit-2
    11 months ago

    If you were on my team and I knew you were doing this, aside from all the other issues everybody else has mentioned in this thread, I’d be going out of my way not just to check every single one of your git commits (both code changes and commit messages) for inaccuracies but also also to find every possible reason to nit pick everything you committed.

    You shouldn’t be using LLMs to write your git commits (messages or code changes.) They hallucinate. But if you are going to use them, you need to spend much more time proofreading what they output than you’d spend writing it yourself. Check every single word for errors. (And, honestly, make the text fit with the way other commits by your team read (assuming you are on a team).)

    In short, if you’re going to use LLMs, DO NOT TRUST ANYTHING THEY GIVE YOU. And don’t be surprised if you get negative blowback from others for using them at all. Keep in mind what can happen if you trust LLMs.

    • Emma @programming.dev
      link
      fedilink
      arrow-up
      2
      ·
      edit-2
      11 months ago

      IMHO, the provided link is largely irrelevant to this topic. It is about lawyers who used ChatGPT as a search engine https://youtu.be/oqSYljRYDEM?t=1436, which is not what it is for, and it will tell you that over and over again. The lawyers in question were not even “trusting ChatGPT”. They blatantly and actively disregarded ChatGPT telling them that it was not a search engine and could not provide legal advice https://youtu.be/oqSYljRYDEM?t=1466

      This topic is about using LLMs to generate natural language describing code changes that it is provided with which is not only completely different than using an LLM as a defacto search engine, but it is also something LLMs are actually meant to do: autocomplete. This topic is more akin to using LLMs to write title headings for legal documents which are already basically complete as is than it is akin to the link provided.

  • jeffhykin@lemm.ee
    link
    fedilink
    arrow-up
    7
    ·
    11 months ago

    Im shocked at all the negativity, this seems like an obvious good usecase to me, and I’m someone who finds most AI predictive stuff useless.

    I never take more than 3 sec on my commit messages, most of them are “fix bug”, “update lib”, “bump”. So it’s a pretty low bar for it to make better messages than mine.

  • lonewalk@lemm.ee
    link
    fedilink
    English
    arrow-up
    6
    ·
    11 months ago

    it’d be interesting to see some examples of what your script came up with - I’m a bit skeptical of what an AI would come up with in terms of a commit message, and I’d think you’d need a pretty complex system to get commit messages to be maximally useful. I’ve found LLMs can stray towards being too high level and struggle if you ask more specific questions.

    but I could also see it as being helpful for a sort of audit log for what changed, and I don’t think it would be too harmful, as long as you’re checking what the LLM is generating and making sure there’s corresponding code changes, that it’s not hallucinating etc.

    hard to tell without examples - perhaps you could expand your post with some?

    an aside, sorry you got such an overwhelmingly toxic response. the amount of angry people on this platform who feel the need to morally educate everyone around them objectively sucks and makes it a really unpleasant place to be.

  • Emma @programming.dev
    link
    fedilink
    arrow-up
    3
    ·
    11 months ago

    Okay, so to be honest, at first, I didn’t understand all the ❌negativity, but I shared this with a friend to get her take on the issue, and she 🕵️‍♀️clued me into the fact that webpack already does this with copilot and pull requests, and the results . . . speak for themselves.

    Wow😅. I didn’t think it would be that bad. It seems that every example I find is just incorrect. I’ll look at the code. It will be a two line change, but the summary will be difficult to follow and often says things that are not part of the code changes. Then, there are also contradictions which make the pull requests harder to follow than if you just read the code with no other context. Darn it. I really thought this was a 🧊cool idea.

    I’m definitely going to be sticking to writing my own commits as always.

  • Kaldo@kbin.social
    link
    fedilink
    arrow-up
    0
    ·
    11 months ago

    If a glorified autocomplete algorithm can write more informative and concise commit messages than you, the actual author behind the code, I think you need to sit down and think long and hard what that actually implies.

    • douglasg14b@programming.dev
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      11 months ago

      And what does it imply?

      That an AI might be better at writing documentation than the average dev, who is largely inept at writing good documentation?

      Understandably, as technical writing isn’t exactly a focus point or career growing thing for most devs. If it was, we would be writing much better code as well.

      I’ve seen my peers work, they could use something like this. I’d welcome it.

  • packadal@beehaw.org
    link
    fedilink
    arrow-up
    0
    ·
    11 months ago

    I’m really curious to see examples of commit messages.

    And more details on how you achieved it, I’d like to give it a go myself, as some of my coworkers commit messages are less than stellar.

    Does it write in the conventional commit formalism ?