Open Side Menu Go to the Top
Register
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** ** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD **

05-09-2017 , 04:29 AM
Look up cascading deletes for your database, will delete child records. Also, as others have mentioned orphaned records shouldn't really exist if you set your relationships up properly - the DB wont permit the delete.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 08:19 AM
Just having a database with primary keys and foreign keys will take care of a lot of data issues. Also, I'm about 99% against using cascades.

****

LL, if I had to take a blind shot at your problem, I'd guess you are running into CRSF token issues, which could happen if you are attempting something that requires log in, you are refreshing a stale resource, you didn't add CRSF params, etc. I'm assuming the file system is properly set up to do read-execute for the user running the system.

If the command prompt output isn't giving any information, find a verbose production error setting, or, find out where the error logs are and do "tail /path/to/logs".

Here's a quick explanation of tail:

https://www.computerhope.com/unix/utail.htm
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 08:57 AM
Nah, most of my apis work, I'm just doing something a little different with one and it isn't working or giving me useful error messages.

99% of the help I've been given is just advice to rename things from singular to plural because of the rails conventions (which I was already following) and stuff like that. I feel like when i talk to someone who knows rails today it's going to take 10 mins to see the error in my way and fix it.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:06 AM
Cascading deletes are like the one place I prefer using triggers to application code.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:08 AM
LL have you tried the Better Errors/Binding of Caller gem combination to get a more granular view of the stack trace?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:17 AM
Quote:
Originally Posted by Larry Legend
Nah, most of my apis work, I'm just doing something a little different with one and it isn't working or giving me useful error messages.

99% of the help I've been given is just advice to rename things from singular to plural because of the rails conventions (which I was already following) and stuff like that. I feel like when i talk to someone who knows rails today it's going to take 10 mins to see the error in my way and fix it.
A quick search found this:

http://stackoverflow.com/questions/2...the-controller

This, and the link to another solution, point to CSRF issues, but ultimately an error inserting in Active Record. Seems there may be a model validation issue as well.

FWIW, I've only had unwanted 500 errors for misconfiged files systems or CSRF issues. The 404 is possibly the state resetting itself and losing the url path, likely a misfired redirect with POST, especially if you are using callbacks to build url params (horrible explanation, but trying express how 404 could result from a CRSF failure).
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:58 AM
So the issue was that I'm an idiot (which was a lock) and I was assigning a has_many relationship through a join table but wasn't setting a relationship directly with the join table.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:58 AM
Quote:
Originally Posted by CBorders
LL have you tried the Better Errors/Binding of Caller gem combination to get a more granular view of the stack trace?
No, but I am definitely going to look into this.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 11:44 AM
Don't use cascade deletes myself as by design I usually go for the never delete anything approach, but don't see what the problem is if the children are dependant on the record you're deleting?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 11:48 AM
I know for a fact some of the largest companies in the database world are not immune to orphaned records, and while it may take several layers of approvals, deleting the bad data is essentially universally considered the play.

The fact you know why it was orphaned answers the major concern.

Unless you need this data and you can do something useful with it, but leaving it to **** things up seems bad.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 12:02 PM
Quote:
Originally Posted by Gullanian
Don't use cascade deletes myself as by design I usually go for the never delete anything approach, but don't see what the problem is if the children are dependant on the record you're deleting?
Reason #1:

Consider this example:

Code:
create table valid_cities (
     city_name varchar primary key
);

create table customers (
      customer_id int primary key,
      city_name varchar,
      foreign key city_name
           references valid_cities (city_name)
);
You may have accidentally misspelled a city, so you delete the city name with the intent to change that and update the customer data. The customer_id also has a bunch of cascading deletes. Ooops.

It's better to manually step through all the changes.

Reason #2:

Cascading deletes are very slow and it's pretty much always faster to just delete manually. Table locking can cause the entire system to pause until the transaction is complete.

This transaction isolation is a major concern. If you are doing deletes step-wise, you can work on smaller transactions that aren't going to cause a long queue of queries.

Reason #3:

I can't think of a single case where I wish I was using cascading deletes. I've always wanted to wrap in transactions, check the work, and make sure my results are correct.

I consider using cascades as dangerous as dropping FKs to make things faster and smoother. It may be faster, but good luck putting all the pieces back together. Constraints are there as a tool, so why not use them?

I have used cascading updates, but honestly, I haven't really found them useful either. I try to consider data static, not dynamic.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 12:08 PM
Quote:
Originally Posted by Larry Legend
So the issue was that I'm an idiot (which was a lock) and I was assigning a has_many relationship through a join table but wasn't setting a relationship directly with the join table.
Well, I was sort of not close on that one, lol.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 12:30 PM
Socket gurus, I have a question. Let's say you have a browser showing a big panel of different system-health widgets (db usage chart, disk space usage bar, active connections, etc.) which you want to keep updated in real time. So like a rudimentary app-dynamics dashboard basically.

Parameters:
  1. You are designing the front and back end.
  2. You will never have 1000s of users hitting this system at once. Maybe 100s in the far-flung future. But for for the foreseeable future, never more than a handful. So too many open sockets on the server is not an issue.
  3. Browser snappiness is very important as this is not just purely internal - client operations people could conceivably see this at some point. Also the boss is ape**** over browser snappiness.
  4. The # of widgets could conceivably grow to a few hundred when you factor in tabs that aren't showing but still being updated. But for the foreseeable future would be more like 5-20. You could conceivably do stuff like only update the active tab - which you may have to do eventually if the app grows huge. But you'd rather not do that for now - as the boss wants to instantly see fresh data everywhere.

Would you:

(A) (naively?) Create a back end socket call and open one socket on the browser for each of these widgets?

(B) Create one open socket - over which the client could initiate (emit) subscribe requests to the back end for whatever it wants? The back end would know how to route these subscribe requests. When something changes in the back-end, data is pushed over the one open socket, and the client knows how to route that data to the correct widget.

If choosing (B), does anyone know if this a common socket pattern? Multi-lane highway socket - heh.


Follow up question - has anyone ever tried to pass a socket through multiple nodes (in this case literally node - but could be any server) back to the browser?

Of course when they developed this thing they went straight from browser to back end. But in our set up there is a node authentication layer that acts as a gateway to the back end - which everything has to hop through. So it's kind of a dumb proxy with authentication, but also a physical barrier to the browser calling any back end end points that we don't want to expose to the internet (microservice to microservice calls).

The REST proxy calls are pretty simple - the authentication node just repeats the REST call from the browser to the specified (via url pattern) back-end microservice node, then passes the response back through to the browser. The only place those get tricky is hopping streams across nodes - which took me a few days to figure out.

However for sockets right now we're terminating a socket in the authentication layer - then it's doing a bunch of business logic to connect sockets to other microservices. The authentication layer then pipes the output from these back-end sockets back to the browser over the single open socket. Ideally I'd like to just pass the whole socket to a back-end microservice which would then have a (functionally if not literally) direct socket connection to the browser.

Last edited by suzzer99; 05-09-2017 at 12:52 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 12:44 PM
I'm doing B for my app and it seems fine. No problems with performance on my $10 DO box and ~30 users. A doesn't seem right at all.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 01:00 PM
Yeah I guess (B) makes sense the more I think about it. Shows how new I am to sockets. If it hasn't been named yet - let's call this pattern - Socket Superhighway.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 02:20 PM
Quote:
Originally Posted by suzzer99
If choosing (B), does anyone know if this a common socket pattern? Multi-lane highway socket - heh.
This is called multiplexing, https://www.rabbitmq.com/blog/2012/0...ng-websockets/

I don't know much about it other than it's a thing.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:01 PM
This is what I have to deal with btw. This little gem of node code is being executed at run time, every time a new socket comes in from the browser.

Code:
const hookSocketHandlers = ( directory, socket ) => {

  let files = fileIo.directoryFilesSync(directory);
  files.forEach( (file) => {
    if( fileIo.isDirectory( directory + '/' + file) ) {
      hookSocketHandlers(directory + '/' + file, socket);
    }
    if( file.indexOf('.js') != -1 ) {
      logger.log('hooking socket handler' + directory + '/' + file);
      let handler = _fs.app_require(directory + '/' + file)(socket);
    }
  });
};
Me: Dude! You cannot have synchronous fileIo happening at run time.

Him: I created this whole app in a week. I haven't had time to optimize it yet.

Today I am still getting push back and being told this is the only way to do it. So I have to fix it myself, make a PR for him to ignore as long as he can, then own it if anything goes wrong. ARGHHHH! The alternative is keep working on the app knowing this time bomb is sitting there.

Oh yeah _fs is his custom version of fs, and app_require is his hack around require - because require won't let you do stuff he wants to do at run time.

Forget it if even makes sense to attach a bunch of handlers to every socket. Maybe this is the best way to do multiplexing?

Last edited by suzzer99; 05-09-2017 at 09:06 PM.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:26 PM
Wat! I have a hard time understanding how it makes sense to iterate over a bunch of arbitrary files to set up handlers...
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:33 PM
Quote:
Originally Posted by suzzer99
Socket gurus, I have a question. Let's say you have a browser showing a big panel of different system-health widgets (db usage chart, disk space usage bar, active connections, etc.) which you want to keep updated in real time. So like a rudimentary app-dynamics dashboard basically.

Parameters:
  1. You are designing the front and back end.
  2. You will never have 1000s of users hitting this system at once. Maybe 100s in the far-flung future. But for for the foreseeable future, never more than a handful. So too many open sockets on the server is not an issue.
  3. Browser snappiness is very important as this is not just purely internal - client operations people could conceivably see this at some point. Also the boss is ape**** over browser snappiness.
  4. The # of widgets could conceivably grow to a few hundred when you factor in tabs that aren't showing but still being updated. But for the foreseeable future would be more like 5-20. You could conceivably do stuff like only update the active tab - which you may have to do eventually if the app grows huge. But you'd rather not do that for now - as the boss wants to instantly see fresh data everywhere.

Would you:

(A) (naively?) Create a back end socket call and open one socket on the browser for each of these widgets?

(B) Create one open socket - over which the client could initiate (emit) subscribe requests to the back end for whatever it wants? The back end would know how to route these subscribe requests. When something changes in the back-end, data is pushed over the one open socket, and the client knows how to route that data to the correct widget.

If choosing (B), does anyone know if this a common socket pattern? Multi-lane highway socket - heh.


Follow up question - has anyone ever tried to pass a socket through multiple nodes (in this case literally node - but could be any server) back to the browser?

Of course when they developed this thing they went straight from browser to back end. But in our set up there is a node authentication layer that acts as a gateway to the back end - which everything has to hop through. So it's kind of a dumb proxy with authentication, but also a physical barrier to the browser calling any back end end points that we don't want to expose to the internet (microservice to microservice calls).

The REST proxy calls are pretty simple - the authentication node just repeats the REST call from the browser to the specified (via url pattern) back-end microservice node, then passes the response back through to the browser. The only place those get tricky is hopping streams across nodes - which took me a few days to figure out.

However for sockets right now we're terminating a socket in the authentication layer - then it's doing a bunch of business logic to connect sockets to other microservices. The authentication layer then pipes the output from these back-end sockets back to the browser over the single open socket. Ideally I'd like to just pass the whole socket to a back-end microservice which would then have a (functionally if not literally) direct socket connection to the browser.
Definitely B.

Also not sure if this is a possibility but I personally would have probably have just used pusher.com to avoid the pain. Would have been cheap for the volume you're talking, then all you have to implement on the server is the rest endpoints. Plus the multiway thing you're describing is baked in. They support both channels and events.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 09:55 PM
lmao. At least make it increment something and write to disk every 1000 connections.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:10 PM
Why is the function a const and files a let when it doesnt get mutated!
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:17 PM
Why do people still use semi-colons, is it out of habit or is there another reason?
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:23 PM
Quote:
Originally Posted by Grue
Why is the function a const and files a let when it doesnt get mutated!
Because he doesn't know JS very well and is working 70 hour weeks developing massive amounts of ill-thought-out new features in a blind panic.

But the awesome part is he goes into a grand mal seizure if I try to touch anything anywhere in his code. Even though I'm the node guy.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:23 PM
Quote:
Originally Posted by Larry Legend
Why do people still use semi-colons, is it out of habit or is there another reason?
Semi-colons are awesome. Compilers like them too.
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote
05-09-2017 , 10:26 PM
Quote:
Originally Posted by blackize5
Wat! I have a hard time understanding how it makes sense to iterate over a bunch of arbitrary files to set up handlers...
I think it's actually a decent pattern. Put some arbitrary number of handlers in a directory, which you can also split into sub-directories as the need arises. Then just glob them to initialize. You don't have to worry about requiring them from any other code - or breaking your require statements when you refactor file or directory names or paths. This removes friction to refactoring file/dir names/paths and makes devs more likely to do it - which is a good thing imo.

I use the pattern in my open source library to auto-initialize the components that handle individual web routes: https://github.com/jackspaniel/nodul...ster/nodule.js

Now what you sure as hell don't do is read over those files every single goddam time at run time. Yeesh.

Here's my refactor that took all of 5 minutes:

Code:
const socketHandlers = [];
const socketHandlerDirectory = '_generated/api/sockets';

glob.sync(socketHandlerDirectory + '/**/*.js',  { cwd: _fs.app_root_path() })
  .forEach(file => {
    logger.log('initializing socket handler' + file);
    socketHandlers.push(_fs.app_require(file));
});

module.exports = (socket) => {
  socketHandlers.forEach(handler => { handler(socket); });
};
** UnhandledExceptionEventHandler :: OFFICIAL LC / CHATTER THREAD ** Quote

      
m