NetBeans Forums

 FAQFAQ   SearchSearch   MemberlistMemberlist   RegisterRegister   ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 
  

C++ Remote Development Issues

 
Post new topic   Reply to topic    NetBeans Forums -> C/C++ Users
View previous topic :: View next topic  
Author Message
elkvis



Joined: 17 Feb 2010
Posts: 8

PostPosted: Fri Jan 13, 2017 10:13 pm    Post subject: C++ Remote Development Issues Reply with quote

I'm using NetBeans 8.2 on Windows 10. My remote build machine is Ubuntu 12.04. My build system is CMake. I have used the following procedure to create my project:

*Clone the git repository locally, on my Windows PC. Project contains 954 .cpp and 176 .h files.
*In NetBeans, Create a new project from existing sources, pointing it at the local copy of the git project
*Create a remote build host, using SSH/SFTP, and assign it as the toolchain for the project.

Doing the initial build works fine. Switching between branches locally does not update any remote files. Editing and saving files locally does not update remote files. The only way remote files seem to get updated is to manually right-click on each file that I have changed, and click "Upload to <user>@xx.xx.xx.xx." Obviously, this stragegy does not help with switching between git branches. For that situation, I must right-click on the project and upload the whole thing to the remote host.

Is it possible to enable some sort of automatic synchronization of files, where saving a file after editing would upload the modified file to the remote host? Is there a way to do the same with all files changed by switching git branches? I've searched for the last several hours, and I've gone through every aspect of IDE and project settings, and come up with nothing. I really hope I'm missing something simple, but there just doesn't seem to be an option to do this. The only thing that comes close is a number of references to PHP projects having an option to do this. It doesn't seem to exist for C++ projects. I've tried "Automatic Copying," "SFTP," and "System level file sharing" modes, and I get varying results.

In SFTP mode:
When I build, it includes the following in the output window:

Code:
Uploading changed files:
    Zipping 4 changed files...
    Uploading zip to <user>@xx.xx.xx.xx...
    Unzipping changed files...


But no files are updated on the remote machine, and the output window shows "BUILD SUCCESSFUL" after doing nothing.

In Automatic Copying mode:
Updated files are sent to the build server, and the build proceeds normally, but the link stage never completes. I can look at the output of ps, and ld is running, and using an expected amount of memory, but shows little to no CPU usage (running the build directly on the remote machine, in an SSH session, ld uses 100% cpu for a minute or so, and then finishes normally).

In System level file sharing mode (using NFS, with the remote machine as the NFS server):
No files are visible in the Netbeans project view, and no build can be done.
Back to top
Vladimir Kvashin
Posted via mailing list.





PostPosted: Sun Jan 15, 2017 8:55 pm    Post subject: C++ Remote Development Issues Reply with quote

Hi!

I'm responsible for NetBeans remote development. Thank you for detailed
story.

The synchronisation (in both SFTP and Auto Copy) *is* automatic - that's
why you can't find an option for such behaviour. All changed files are
uploaded to remote before building.

I think the problem is in time difference between the two machines.

The fact that it does not build in "System level file sharing" mode, i.
e. when does nothing with synchronization, but make still does nothing
proves that.

The workarounds are two: 1) sync time; 2) use "clean & build" instead of
just "build"

Vladimir


On 01/14/2017 01:13 AM, elkvis wrote:
Quote:
I'm using NetBeans 8.2 on Windows 10. My remote build machine is Ubuntu 12.04. My build system is CMake. I have used the following procedure to create my project:

*Clone the git repository locally, on my Windows PC. Project contains 954 .cpp and 176 .h files.
*In NetBeans, Create a new project from existing sources, pointing it at the local copy of the git project
*Create a remote build host, using SSH/SFTP, and assign it as the toolchain for the project.

Doing the initial build works fine. Switching between branches locally does not update any remote files. Editing and saving files locally does not update remote files. The only way remote files seem to get updated is to manually right-click on each file that I have changed, and click "Upload to <user>@xx.xx.xx.xx." Obviously, this stragegy does not help with switching between git branches. For that situation, I must right-click on the project and upload the whole thing to the remote host.

Is it possible to enable some sort of automatic synchronization of files, where saving a file after editing would upload the modified file to the remote host? Is there a way to do the same with all files changed by switching git branches? I've searched for the last several hours, and I've gone through every aspect of IDE and project settings, and come up with nothing. I really hope I'm missing something simple, but there just doesn't seem to be an option to do this. The only thing that comes close is a number of references to PHP projects having an option to do this. It doesn't seem to exist for C++ projects. I've tried "Automatic Copying," "SFTP," and "System level file sharing" modes, and I get varying results.

In SFTP mode:
When I build, it includes the following in the output window:


Code:
Uploading changed files:
Zipping 4 changed files...
Uploading zip to <user>@xx.xx.xx.xx...
Unzipping changed files...



But no files are updated on the remote machine, and the output window shows "BUILD SUCCESSFUL" after doing nothing.

In Automatic Copying mode:
Updated files are sent to the build server, and the build proceeds normally, but the link stage never completes. I can look at the output of ps, and ld is running, and using an expected amount of memory, but shows little to no CPU usage (running the build directly on the remote machine, in an SSH session, ld uses 100% cpu for a minute or so, and then finishes normally).

In System level file sharing mode (using NFS, with the remote machine as the NFS server):
No files are copied, and the behavior is essentially the same as SFTP mode, with no noticeable differences.



Back to top
elkvis



Joined: 17 Feb 2010
Posts: 8

PostPosted: Mon Jan 16, 2017 2:01 pm    Post subject: Reply with quote

Quote:
I think the problem is in time difference between the two machines.


The windows machine and the remote linux machine are within 1 second of being in perfect sync. I don't think I can demand much more than that.

Quote:
The workarounds are two: 1) sync time; 2) use "clean & build" instead of
just "build"


1) See above. 2) Clean and build is an option, but not a good one, because even with a parallel build across 24 cores, it's an 8-10 minute build, depending on the remote machine's load.
Back to top
elkvis



Joined: 17 Feb 2010
Posts: 8

PostPosted: Tue Jan 17, 2017 2:00 pm    Post subject: Reply with quote

Update!

I found that, if I save the files, wait a few seconds, then initiate a build, it works just fine. However, if I do not save the files first, but instead depend on the save-on-build to do it for me, then no files are copied.

I think you were right about the time sync issue. My windows machine is on an active directory domain, so I tried syncing the remote build machine to the DC's time, and now they're even closer in time, but I still have to save first, wait a few seconds, then build. Still much easier and more consistent than hand copying changed files.
Back to top
Vladimir Kvashin
Posted via mailing list.





PostPosted: Tue Jan 24, 2017 7:47 am    Post subject: C++ Remote Development Issues Reply with quote

Let us double check. You wrote that you also tried the "System level
file sharing" mode. And that in this mode the same problem exists. Is
that true?

In "System level file sharing" mode IDE just saves files exactly as it
does when you work with your local host and then launches make (or what
is mentioned in your project options) - that's it. So if in "System
level file sharing" mode problems are the same, then the cause is not in
uploading files to the remote host, but in something different. Most
likely it is not related with IDE at all.

I'd also like to note, that if your files reside not on the machine you
launch your build, but in some network location, then time on this file
system may differ from both local and remote time. I faced such
situations a couple of times...

Vladimir


On 01/17/2017 05:00 PM, elkvis wrote:
Quote:
Update!

I found that, if I save the files, wait a few seconds, then initiate a build, it works just fine. However, if I do not save the files first, but instead depend on the save-on-build to do it for me, then no files are copied.

I think you were right about the time sync issue. My windows machine is on an active directory domain, so I tried syncing the remote build machine to the DC's time, and now they're even closer in time, but I still have to save first, wait a few seconds, then build. Still much easier and more consistent than hand copying changed files.



Back to top
elkvis



Joined: 17 Feb 2010
Posts: 8

PostPosted: Tue Jan 24, 2017 1:01 pm    Post subject: Re: C++ Remote Development Issues Reply with quote

Vladimir Kvashin wrote:
Let us double check. You wrote that you also tried the "System level
file sharing" mode. And that in this mode the same problem exists. Is
that true?


It's not exactly the same in "System level file sharing" mode. In that mode, when I try to open a remote project, I can't see any files on my local machine. Maybe I'm missing a step. Perhaps you could clear it up for me. Do you know of a complete step-by-step guide to setting up NFS-based remote building between a Windows machine running NetBeans, and a Linux remote build host?

Of all the modes, "Automatic copying" seems to work the best, except for the fact that the linking step never ever completes. Refer to my initial post for the details on that. Is it possible that the size of the final binary (>1GB, including debugging info) is causing this problem? Is it trying to copy the binary to my local machine, from the build host?
Back to top
Vladimir Kvashin
Posted via mailing list.





PostPosted: Mon Jan 30, 2017 12:36 pm    Post subject: C++ Remote Development Issues Reply with quote

The "System level file sharing" mode works only if your project and sources reside in a place that is accessible from both local and remote host. If it's not the case, then this mode does not make sense at all. I mentioned this mode only because as I understood it has the same problem. If it does not work for your case, let's leave this mode and discuss other modes.
I recommend using "SFTP" mode. It's more reliable and straightforward than "Auto Copy". I can't believe that "Auto Copy" works better than "SFTP".
Could you do the following experiment.
Select "SFTP" mode, change a file, press "Build".
Please check whether IDE "thinks" it uploads this file (something like below one should appear in build window)



term.png
 Description:
 Filesize:  8.37 KB
 Viewed:  856 Time(s)

term.png


Back to top
elkvis



Joined: 17 Feb 2010
Posts: 8

PostPosted: Mon Jan 30, 2017 2:12 pm    Post subject: Re: C++ Remote Development Issues Reply with quote

Vladimir Kvashin wrote:
Could you do the following experiment.
Select "SFTP" mode, change a file, press "Build".
Please check whether IDE "thinks" it uploads this file (something like below one should appear in build window)


Sometimes it does, and other times it does not. It is inconsistent, and impossible to determine when it will work, and when it will not. The only times that it seems to work are when I clean and build, like you suggested, and when I switch git branches (I was mistaken about that in my initial post). I have synchronized the time between the windows machine and the build machine to the best of my ability, but it didn't improve the situation at all.

Is there anything I can do on the build host, to see the temporary files generated when NetBeans zips up the changed files, and sends them to the build host?
Back to top
Display posts from previous:   
Post new topic   Reply to topic    NetBeans Forums -> C/C++ Users All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
You cannot attach files in this forum
You can download files in this forum


Powered by phpBB
By use of this website, you agree to the NetBeans Policies and Terms of Use. © 2012, Oracle Corporation and/or its affiliates. Sponsored by Oracle logo