Home → Problems →

Non standard YAML format?

• Edit

Frank Koehl

15 Apr, 2013 05:34 PM

When it comes to file change lists in the changeset API calls, what kind of data structure are you trying to represent? I couldn't figure out how to get my YAML emitter to create a list in the format you specified, so I had to build the list by hand (it feels so dirty...).

# Never seen this before
- - M
  - /trunk/test/unit/changeset_test.rb
- - M
  - /trunk/app/models/changeset.rb
- - M
  - /trunk/db/schema.rb

# Shouldn't it be more like this?
M:
  - /trunk/test/unit/changeset_test.rb
  - /trunk/app/models/changeset.rb
  - /trunk/db/schema.rb

Hell with it, I'm going to ask the obvious question: Why are you using YAML for this at all? First, one standard (YAML) inside another (XML)? Yuck. Second, why can't I just send you the raw output ...

  M /trunk/test/unit/changeset_test.rb
  M /trunk/app/models/changeset.rb
  M /trunk/db/schema.rb

that every SCM tool spits out by default and let you parse it into whatever voodoo format you want/need for your systems? Seems a little backwards.

P.S. The documented YAML formatting for the "Create Changeset" API call does not match what is actually required. Just a heads up.

1. Support Staff 1 Posted by Tiger Team on 15 Apr, 2013 06:01 PM

   

   Hey Frank,

   No idea. You're looking at code probably from 2006. But here's how Github posts their changesets to us, written by the same guy who wrote the changeset code:

   https://github.com/github/github-services/blob/master/lib/services/...

   • Edit

2. Support Staff 2 Posted by Tiger Team on 15 Apr, 2013 06:07 PM

   

   I just did a bit of digging for you; you're right, it really should just parse a chunk of text.

   • Edit

3. 3 Posted by Frank Koehl on 15 Apr, 2013 06:17 PM

   

   Courtenay, thanks for digging. Good to know I'm not the crazy one.

   So the guy who wrote the GitHub integration was responsible for the original spec for changesets within Lighthouse? As in he used to work for entp? You guys are probably better off... ;-)

   (Edit) So when can I expect to see an update eliminating these YAML hoops?

   • Edit

4. 4 Posted by Julien on 16 Apr, 2013 12:41 AM

   

   Hey Frank,

   You can try and provide the simpler version:

   A somefile
   M otherfile
   ...
   

   Let me know how it goes.

   • Edit

5. 5 Posted by Frank Koehl on 16 Apr, 2013 01:20 PM

   

   Thanks, Julien, that almost worked flawlessly.

   My example was too simple. Git and Subversion both produce the file list with multiple spaces between the status character (A, M, D, R, etc...) and the file name. In other words, the modified file list is not restricted to a single space. Your implementation requires a single space, when it should look for any number of uninterrupted spaces.

   # My original example
   M /trunk/test/unit/changeset_test.rb
   M /trunk/app/models/changeset.rb
   M /trunk/db/schema.rb
   
   # A more accurate representation
   M       /trunk/test/unit/changeset_test.rb
   M       /trunk/app/models/changeset.rb
   M       /trunk/db/schema.rb
   

   Here's the Git command to produce the list I described above...

   git diff-tree --no-commit-id --name-status -r OLDREV NEWREV
   

   • Edit

6. 6 Posted by Julien on 16 Apr, 2013 05:25 PM

   

   Mmm. Actually the code should already handle this case and I have tests for it. Do you have an example that failed in your account that I can look at?

   Thanks!

   • Edit

7. 7 Posted by Frank Koehl on 16 Apr, 2013 06:52 PM

   

   Sure, here's a broken changeset.

   Should have 2 files in the change list at the bottom...

   M       meta/git-hooks/origin-master-post-receive
   A       meta/git-hooks/post-receive
   

   • Edit

8. 8 Posted by Julien on 17 Apr, 2013 08:43 PM

   

   Hey Frank,

   I deployed a couple of fixes for this. It should now work as expected with both the old format (yaml) and the new one (regular SCM output).

   Let me know how it goes.

   • Edit

9. 9 Posted by Frank Koehl on 18 Apr, 2013 01:28 PM

   

   Now requests are rejected with the YAML formatting error from before...

   Changes should be an array of arrays of strings like this:
   "---\n- - M\n  - /foo/bar/baz\n- - A\n  - /foo\n"
   

   The "raw" relevant portion of the XML request looks like this...

   <changes>
   M       meta/git-hooks/origin-master-post-receive
   A       meta/git-hooks/post-receive
   </changes>
   

   • Edit

10. 10 Posted by Thorsteinn Yngv... on 18 Apr, 2013 02:23 PM

    

    Hi Julien,

    Is it possible that the changes you deployed broke our GitHub hook? We haven't been able to push since around the time of your deployment.

    • Edit

11. 11 Posted by Julien on 18 Apr, 2013 03:16 PM

    

    Hi Frank,

    I will look into it again.

    Thorsteinn: I don't think so. We use Github ourselves and have pushed many changes after the deploy and I made sure that they still work. Also, the hook presumably happens separately from the code push, and you should still be able to push code to Github even if all your hooks are broken. Or maybe you are not using the normal Lighthouse Github hook, and use a custom one? Can you tell me a bit more about your setup?

    • Edit

12. 12 Posted by Borgar on 18 Apr, 2013 05:32 PM

    

    Hi, I am a developer on Thorsteinn's project and can tell you that:

    1. We have had no commit syncing from GitHub to Lighthouse today (last push payload according to Lighthouse was: April 17th, 2013 @ 11:00 PM) despite several pushes (admittedly few).

    2. AFAIK, we use a standard setup through a GitHub service hook. Nothing has been changed recently. Running the "test hook" at GutHub reports "Payload deployed" - whatever that means.

    3. Our Lighthouse project ID is 48730.

    4. I've just tried pushing and the changeset is not turning up in Lighthouse. The hash is a1015c752d69771c33e955230921d7dd5e29273e.

    • Edit

13. 13 Posted by Julien on 18 Apr, 2013 06:05 PM

    

    Hi Borgar,

    I'm taking a look.

    • Edit

14. 14 Posted by Julien on 18 Apr, 2013 06:37 PM

    

    Hi Frank,

    Your issues should be solved now.

    Borgar: it seems the problem is coming from Github as the post-commit hook is not working on some repositories: they aren't even making the call. I have contacted them and I'm waiting to hear back. I'll keep you posted.

    • Edit

15. 15 Posted by Frank Koehl on 18 Apr, 2013 07:00 PM

    

    Confirmed, hook is working as expected.

    Thanks for the quick turnaround, Julien!

    • Edit

16. 16 Posted by Julien on 19 Apr, 2013 10:52 PM

    

    Hi Borgar,

    I think we fixed it. Give it a try and let me know how it goes.

    • Edit

17. 17 Posted by Borgar on 22 Apr, 2013 11:42 AM

    

    Yep, also think this is fixed. Top item in our feed is from GitHub. Thanks!

    • Edit

18. 18 Posted by Borgar on 22 Apr, 2013 12:44 PM

    

    This is working for us too now. Thanks a lot!

    • Edit

19. Julien closed this discussion on 22 Apr, 2013 05:27 PM.