Non standard YAML format?

Frank Koehl's Avatar

Frank Koehl

15 Apr, 2013 05:34 PM

When it comes to file change lists in the changeset API calls, what kind of data structure are you trying to represent? I couldn't figure out how to get my YAML emitter to create a list in the format you specified, so I had to build the list by hand (it feels so dirty...).

# Never seen this before
- - M
  - /trunk/test/unit/changeset_test.rb
- - M
  - /trunk/app/models/changeset.rb
- - M
  - /trunk/db/schema.rb

# Shouldn't it be more like this?
M:
  - /trunk/test/unit/changeset_test.rb
  - /trunk/app/models/changeset.rb
  - /trunk/db/schema.rb

Hell with it, I'm going to ask the obvious question: Why are you using YAML for this at all? First, one standard (YAML) inside another (XML)? Yuck. Second, why can't I just send you the raw output ...

  M /trunk/test/unit/changeset_test.rb
  M /trunk/app/models/changeset.rb
  M /trunk/db/schema.rb

that every SCM tool spits out by default and let you parse it into whatever voodoo format you want/need for your systems? Seems a little backwards.

P.S. The documented YAML formatting for the "Create Changeset" API call does not match what is actually required. Just a heads up.

  1. Support Staff 1 Posted by Tiger Team on 15 Apr, 2013 06:01 PM

    Tiger Team's Avatar

    Hey Frank,

    No idea. You're looking at code probably from 2006. But here's how Github posts their changesets to us, written by the same guy who wrote the changeset code:

    https://github.com/github/github-services/blob/master/lib/services/...

  2. Support Staff 2 Posted by Tiger Team on 15 Apr, 2013 06:07 PM

    Tiger Team's Avatar

    I just did a bit of digging for you; you're right, it really should just parse a chunk of text.

  3. 3 Posted by Frank Koehl on 15 Apr, 2013 06:17 PM

    Frank Koehl's Avatar

    Courtenay, thanks for digging. Good to know I'm not the crazy one.

    So the guy who wrote the GitHub integration was responsible for the original spec for changesets within Lighthouse? As in he used to work for entp? You guys are probably better off... ;-)

    (Edit) So when can I expect to see an update eliminating these YAML hoops?

  4. 4 Posted by Julien on 16 Apr, 2013 12:41 AM

    Julien's Avatar

    Hey Frank,

    You can try and provide the simpler version:

    A somefile
    M otherfile
    ...
    

    Let me know how it goes.

  5. 5 Posted by Frank Koehl on 16 Apr, 2013 01:20 PM

    Frank Koehl's Avatar

    Thanks, Julien, that almost worked flawlessly.

    My example was too simple. Git and Subversion both produce the file list with multiple spaces between the status character (A, M, D, R, etc...) and the file name. In other words, the modified file list is not restricted to a single space. Your implementation requires a single space, when it should look for any number of uninterrupted spaces.

    # My original example
    M /trunk/test/unit/changeset_test.rb
    M /trunk/app/models/changeset.rb
    M /trunk/db/schema.rb
    
    # A more accurate representation
    M       /trunk/test/unit/changeset_test.rb
    M       /trunk/app/models/changeset.rb
    M       /trunk/db/schema.rb
    

    Here's the Git command to produce the list I described above...

    git diff-tree --no-commit-id --name-status -r OLDREV NEWREV
    
  6. 6 Posted by Julien on 16 Apr, 2013 05:25 PM

    Julien's Avatar

    Mmm. Actually the code should already handle this case and I have tests for it. Do you have an example that failed in your account that I can look at?

    Thanks!

  7. 7 Posted by Frank Koehl on 16 Apr, 2013 06:52 PM

    Frank Koehl's Avatar

    Sure, here's a broken changeset.

    Should have 2 files in the change list at the bottom...

    M       meta/git-hooks/origin-master-post-receive
    A       meta/git-hooks/post-receive
    
  8. 8 Posted by Julien on 17 Apr, 2013 08:43 PM

    Julien's Avatar

    Hey Frank,

    I deployed a couple of fixes for this. It should now work as expected with both the old format (yaml) and the new one (regular SCM output).

    Let me know how it goes.

  9. 9 Posted by Frank Koehl on 18 Apr, 2013 01:28 PM

    Frank Koehl's Avatar

    Now requests are rejected with the YAML formatting error from before...

    Changes should be an array of arrays of strings like this:
    "---\n- - M\n  - /foo/bar/baz\n- - A\n  - /foo\n"
    

    The "raw" relevant portion of the XML request looks like this...

    <changes>
    M       meta/git-hooks/origin-master-post-receive
    A       meta/git-hooks/post-receive
    </changes>
    
  10. 10 Posted by Thorsteinn Yngv... on 18 Apr, 2013 02:23 PM

    Thorsteinn Yngvi Gudmundsson's Avatar

    Hi Julien,

    Is it possible that the changes you deployed broke our GitHub hook? We haven't been able to push since around the time of your deployment.

  11. 11 Posted by Julien on 18 Apr, 2013 03:16 PM

    Julien's Avatar

    Hi Frank,

    I will look into it again.

    Thorsteinn: I don't think so. We use Github ourselves and have pushed many changes after the deploy and I made sure that they still work. Also, the hook presumably happens separately from the code push, and you should still be able to push code to Github even if all your hooks are broken. Or maybe you are not using the normal Lighthouse Github hook, and use a custom one? Can you tell me a bit more about your setup?

  12. 12 Posted by Borgar on 18 Apr, 2013 05:32 PM

    Borgar's Avatar

    Hi, I am a developer on Thorsteinn's project and can tell you that:

    1. We have had no commit syncing from GitHub to Lighthouse today (last push payload according to Lighthouse was: April 17th, 2013 @ 11:00 PM) despite several pushes (admittedly few).

    2. AFAIK, we use a standard setup through a GitHub service hook. Nothing has been changed recently. Running the "test hook" at GutHub reports "Payload deployed" - whatever that means.

    3. Our Lighthouse project ID is 48730.

    4. I've just tried pushing and the changeset is not turning up in Lighthouse. The hash is a1015c752d69771c33e955230921d7dd5e29273e.

  13. 13 Posted by Julien on 18 Apr, 2013 06:05 PM

    Julien's Avatar

    Hi Borgar,

    I'm taking a look.

  14. 14 Posted by Julien on 18 Apr, 2013 06:37 PM

    Julien's Avatar

    Hi Frank,

    Your issues should be solved now.

    Borgar: it seems the problem is coming from Github as the post-commit hook is not working on some repositories: they aren't even making the call. I have contacted them and I'm waiting to hear back. I'll keep you posted.

  15. 15 Posted by Frank Koehl on 18 Apr, 2013 07:00 PM

    Frank Koehl's Avatar

    Confirmed, hook is working as expected.

    Thanks for the quick turnaround, Julien!

  16. 16 Posted by Julien on 19 Apr, 2013 10:52 PM

    Julien's Avatar

    Hi Borgar,

    I think we fixed it. Give it a try and let me know how it goes.

  17. 17 Posted by Borgar on 22 Apr, 2013 11:42 AM

    Borgar's Avatar

    Yep, also think this is fixed. Top item in our feed is from GitHub. Thanks!

  18. 18 Posted by Borgar on 22 Apr, 2013 12:44 PM

    Borgar's Avatar

    This is working for us too now. Thanks a lot!

  19. Julien closed this discussion on 22 Apr, 2013 05:27 PM.

Discussions are closed to public comments.
If you need help with Lighthouse please start a new discussion.

Keyboard shortcuts

Generic

? Show this help
ESC Blurs the current field

Comment Form

r Focus the comment reply box
^ + ↩ Submit the comment

You can use Command ⌘ instead of Control ^ on Mac